2023-02-11 19:53:56,472 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-11 19:53:56,731 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4 2023-02-11 19:53:56,749 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-11 19:53:56,750 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3, deleteOnExit=true 2023-02-11 19:53:56,750 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-11 19:53:56,750 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/test.cache.data in system properties and HBase conf 2023-02-11 19:53:56,751 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/hadoop.tmp.dir in system properties and HBase conf 2023-02-11 19:53:56,751 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/hadoop.log.dir in system properties and HBase conf 2023-02-11 19:53:56,752 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-11 19:53:56,752 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-11 19:53:56,752 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-11 19:53:56,886 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-11 19:53:57,283 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-11 19:53:57,287 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-11 19:53:57,287 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-11 19:53:57,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-11 19:53:57,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 19:53:57,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-11 19:53:57,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-11 19:53:57,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 19:53:57,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 19:53:57,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-11 19:53:57,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/nfs.dump.dir in system properties and HBase conf 2023-02-11 19:53:57,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/java.io.tmpdir in system properties and HBase conf 2023-02-11 19:53:57,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 19:53:57,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-11 19:53:57,291 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-11 19:53:57,775 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 19:53:57,779 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 19:53:58,060 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-11 19:53:58,235 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-11 19:53:58,250 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 19:53:58,286 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 19:53:58,320 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/java.io.tmpdir/Jetty_localhost_45937_hdfs____.51lyqi/webapp 2023-02-11 19:53:58,480 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45937 2023-02-11 19:53:58,492 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 19:53:58,492 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 19:53:58,878 WARN [Listener at localhost/42039] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 19:53:58,982 WARN [Listener at localhost/42039] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 19:53:58,999 WARN [Listener at localhost/42039] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 19:53:59,006 INFO [Listener at localhost/42039] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 19:53:59,010 INFO [Listener at localhost/42039] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/java.io.tmpdir/Jetty_localhost_45513_datanode____1mutwd/webapp 2023-02-11 19:53:59,104 INFO [Listener at localhost/42039] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45513 2023-02-11 19:53:59,410 WARN [Listener at localhost/40035] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 19:53:59,422 WARN [Listener at localhost/40035] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 19:53:59,426 WARN [Listener at localhost/40035] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 19:53:59,428 INFO [Listener at localhost/40035] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 19:53:59,434 INFO [Listener at localhost/40035] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/java.io.tmpdir/Jetty_localhost_38327_datanode____icmo4a/webapp 2023-02-11 19:53:59,531 INFO [Listener at localhost/40035] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38327 2023-02-11 19:53:59,542 WARN [Listener at localhost/41441] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 19:53:59,558 WARN [Listener at localhost/41441] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 19:53:59,562 WARN [Listener at localhost/41441] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 19:53:59,563 INFO [Listener at localhost/41441] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 19:53:59,569 INFO [Listener at localhost/41441] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/java.io.tmpdir/Jetty_localhost_33361_datanode____.eskf9n/webapp 2023-02-11 19:53:59,668 INFO [Listener at localhost/41441] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33361 2023-02-11 19:53:59,680 WARN [Listener at localhost/38771] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 19:54:00,056 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x369b18a582ffd9ab: Processing first storage report for DS-0daac90e-51bb-4956-8b19-2ed925bb679b from datanode 1c013e61-8b4b-4f0a-8efd-0dfb5b64fb4d 2023-02-11 19:54:00,058 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x369b18a582ffd9ab: from storage DS-0daac90e-51bb-4956-8b19-2ed925bb679b node DatanodeRegistration(127.0.0.1:38329, datanodeUuid=1c013e61-8b4b-4f0a-8efd-0dfb5b64fb4d, infoPort=33943, infoSecurePort=0, ipcPort=41441, storageInfo=lv=-57;cid=testClusterID;nsid=697543106;c=1676145237849), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-11 19:54:00,058 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa3cc97d324270212: Processing first storage report for DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342 from datanode 1d17abb3-77b3-4fd1-a223-1cc76825d5c0 2023-02-11 19:54:00,058 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa3cc97d324270212: from storage DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342 node DatanodeRegistration(127.0.0.1:37117, datanodeUuid=1d17abb3-77b3-4fd1-a223-1cc76825d5c0, infoPort=33187, infoSecurePort=0, ipcPort=40035, storageInfo=lv=-57;cid=testClusterID;nsid=697543106;c=1676145237849), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 19:54:00,058 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x369b18a582ffd9ab: Processing first storage report for DS-3250e948-05ab-451e-b3ab-d31108d735d6 from datanode 1c013e61-8b4b-4f0a-8efd-0dfb5b64fb4d 2023-02-11 19:54:00,058 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x369b18a582ffd9ab: from storage DS-3250e948-05ab-451e-b3ab-d31108d735d6 node DatanodeRegistration(127.0.0.1:38329, datanodeUuid=1c013e61-8b4b-4f0a-8efd-0dfb5b64fb4d, infoPort=33943, infoSecurePort=0, ipcPort=41441, storageInfo=lv=-57;cid=testClusterID;nsid=697543106;c=1676145237849), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 19:54:00,058 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa3cc97d324270212: Processing first storage report for DS-e0494eff-05c2-463e-92a2-f62d9007cd4a from datanode 1d17abb3-77b3-4fd1-a223-1cc76825d5c0 2023-02-11 19:54:00,059 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa3cc97d324270212: from storage DS-e0494eff-05c2-463e-92a2-f62d9007cd4a node DatanodeRegistration(127.0.0.1:37117, datanodeUuid=1d17abb3-77b3-4fd1-a223-1cc76825d5c0, infoPort=33187, infoSecurePort=0, ipcPort=40035, storageInfo=lv=-57;cid=testClusterID;nsid=697543106;c=1676145237849), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-11 19:54:00,064 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfb10e16a5c738bb: Processing first storage report for DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c from datanode 2f25de57-daa1-42aa-aa7e-60a3bc1a94d0 2023-02-11 19:54:00,064 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfb10e16a5c738bb: from storage DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c node DatanodeRegistration(127.0.0.1:44801, datanodeUuid=2f25de57-daa1-42aa-aa7e-60a3bc1a94d0, infoPort=33805, infoSecurePort=0, ipcPort=38771, storageInfo=lv=-57;cid=testClusterID;nsid=697543106;c=1676145237849), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 19:54:00,064 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xfb10e16a5c738bb: Processing first storage report for DS-de0da92e-138d-4e06-8a13-b2623d9151c6 from datanode 2f25de57-daa1-42aa-aa7e-60a3bc1a94d0 2023-02-11 19:54:00,064 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xfb10e16a5c738bb: from storage DS-de0da92e-138d-4e06-8a13-b2623d9151c6 node DatanodeRegistration(127.0.0.1:44801, datanodeUuid=2f25de57-daa1-42aa-aa7e-60a3bc1a94d0, infoPort=33805, infoSecurePort=0, ipcPort=38771, storageInfo=lv=-57;cid=testClusterID;nsid=697543106;c=1676145237849), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 19:54:00,175 DEBUG [Listener at localhost/38771] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4 2023-02-11 19:54:00,242 INFO [Listener at localhost/38771] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/zookeeper_0, clientPort=50506, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-11 19:54:00,257 INFO [Listener at localhost/38771] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=50506 2023-02-11 19:54:00,267 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:00,270 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:00,541 INFO [Listener at localhost/38771] util.FSUtils(479): Created version file at hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f with version=8 2023-02-11 19:54:00,541 INFO [Listener at localhost/38771] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/hbase-staging 2023-02-11 19:54:00,827 INFO [Listener at localhost/38771] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-11 19:54:01,286 INFO [Listener at localhost/38771] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 19:54:01,317 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,318 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,318 INFO [Listener at localhost/38771] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 19:54:01,318 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,319 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 19:54:01,457 INFO [Listener at localhost/38771] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 19:54:01,531 DEBUG [Listener at localhost/38771] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-11 19:54:01,625 INFO [Listener at localhost/38771] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37191 2023-02-11 19:54:01,635 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:01,638 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:01,657 INFO [Listener at localhost/38771] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37191 connecting to ZooKeeper ensemble=127.0.0.1:50506 2023-02-11 19:54:01,695 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:371910x0, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 19:54:01,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37191-0x10044f292690000 connected 2023-02-11 19:54:01,722 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 19:54:01,723 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 19:54:01,726 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 19:54:01,733 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37191 2023-02-11 19:54:01,734 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37191 2023-02-11 19:54:01,734 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37191 2023-02-11 19:54:01,735 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37191 2023-02-11 19:54:01,736 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37191 2023-02-11 19:54:01,741 INFO [Listener at localhost/38771] master.HMaster(439): hbase.rootdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f, hbase.cluster.distributed=false 2023-02-11 19:54:01,807 INFO [Listener at localhost/38771] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 19:54:01,807 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,807 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,807 INFO [Listener at localhost/38771] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 19:54:01,807 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,808 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 19:54:01,812 INFO [Listener at localhost/38771] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 19:54:01,815 INFO [Listener at localhost/38771] ipc.NettyRpcServer(120): Bind to /172.31.14.131:32837 2023-02-11 19:54:01,817 INFO [Listener at localhost/38771] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 19:54:01,823 DEBUG [Listener at localhost/38771] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 19:54:01,824 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:01,826 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:01,827 INFO [Listener at localhost/38771] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:32837 connecting to ZooKeeper ensemble=127.0.0.1:50506 2023-02-11 19:54:01,831 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:328370x0, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 19:54:01,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:32837-0x10044f292690001 connected 2023-02-11 19:54:01,832 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 19:54:01,834 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 19:54:01,835 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 19:54:01,835 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=32837 2023-02-11 19:54:01,836 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=32837 2023-02-11 19:54:01,836 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=32837 2023-02-11 19:54:01,840 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=32837 2023-02-11 19:54:01,840 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=32837 2023-02-11 19:54:01,852 INFO [Listener at localhost/38771] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 19:54:01,853 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,853 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,853 INFO [Listener at localhost/38771] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 19:54:01,853 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,854 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 19:54:01,854 INFO [Listener at localhost/38771] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 19:54:01,855 INFO [Listener at localhost/38771] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43747 2023-02-11 19:54:01,856 INFO [Listener at localhost/38771] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 19:54:01,857 DEBUG [Listener at localhost/38771] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 19:54:01,858 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:01,860 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:01,863 INFO [Listener at localhost/38771] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43747 connecting to ZooKeeper ensemble=127.0.0.1:50506 2023-02-11 19:54:01,868 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:437470x0, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 19:54:01,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43747-0x10044f292690002 connected 2023-02-11 19:54:01,869 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 19:54:01,870 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 19:54:01,871 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 19:54:01,871 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43747 2023-02-11 19:54:01,876 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43747 2023-02-11 19:54:01,876 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43747 2023-02-11 19:54:01,877 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43747 2023-02-11 19:54:01,878 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43747 2023-02-11 19:54:01,890 INFO [Listener at localhost/38771] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 19:54:01,890 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,890 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,891 INFO [Listener at localhost/38771] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 19:54:01,891 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 19:54:01,891 INFO [Listener at localhost/38771] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 19:54:01,891 INFO [Listener at localhost/38771] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 19:54:01,893 INFO [Listener at localhost/38771] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42733 2023-02-11 19:54:01,893 INFO [Listener at localhost/38771] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 19:54:01,896 DEBUG [Listener at localhost/38771] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 19:54:01,897 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:01,899 INFO [Listener at localhost/38771] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:01,900 INFO [Listener at localhost/38771] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42733 connecting to ZooKeeper ensemble=127.0.0.1:50506 2023-02-11 19:54:01,903 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:427330x0, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 19:54:01,905 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:42733-0x10044f292690003 connected 2023-02-11 19:54:01,905 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 19:54:01,905 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 19:54:01,906 DEBUG [Listener at localhost/38771] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 19:54:01,907 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42733 2023-02-11 19:54:01,907 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42733 2023-02-11 19:54:01,907 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42733 2023-02-11 19:54:01,910 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42733 2023-02-11 19:54:01,912 DEBUG [Listener at localhost/38771] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42733 2023-02-11 19:54:01,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:54:01,925 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 19:54:01,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:54:01,943 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 19:54:01,944 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 19:54:01,943 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 19:54:01,943 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 19:54:01,945 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:01,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 19:54:01,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37191,1676145240670 from backup master directory 2023-02-11 19:54:01,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 19:54:01,951 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:54:01,951 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 19:54:01,951 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 19:54:01,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:54:01,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-11 19:54:01,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-11 19:54:02,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/hbase.id with ID: f498648a-9920-464d-aa8f-16d050ee37cb 2023-02-11 19:54:02,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 19:54:02,116 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:02,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x29489d83 to 127.0.0.1:50506 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 19:54:02,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47b4d3d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 19:54:02,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 19:54:02,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-11 19:54:02,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-11 19:54:02,233 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-11 19:54:02,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:54:02,238 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:54:02,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 19:54:02,272 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store-tmp 2023-02-11 19:54:02,307 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:02,307 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 19:54:02,307 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 19:54:02,307 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 19:54:02,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 19:54:02,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 19:54:02,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 19:54:02,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676145242307Disabling compacts and flushes for region at 1676145242307Disabling writes for close at 1676145242308 (+1 ms)Writing region close event to WAL at 1676145242308Closed at 1676145242308 2023-02-11 19:54:02,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/WALs/jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:54:02,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37191%2C1676145240670, suffix=, logDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/WALs/jenkins-hbase4.apache.org,37191,1676145240670, archiveDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/oldWALs, maxLogs=10 2023-02-11 19:54:02,383 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK] 2023-02-11 19:54:02,383 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK] 2023-02-11 19:54:02,383 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK] 2023-02-11 19:54:02,391 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:54:02,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/WALs/jenkins-hbase4.apache.org,37191,1676145240670/jenkins-hbase4.apache.org%2C37191%2C1676145240670.1676145242339 2023-02-11 19:54:02,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK], DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK], DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK]] 2023-02-11 19:54:02,456 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:02,456 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:02,460 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 19:54:02,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 19:54:02,519 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-11 19:54:02,525 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-11 19:54:02,548 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-11 19:54:02,560 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:02,567 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 19:54:02,568 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 19:54:02,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 19:54:02,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:02,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73980036, jitterRate=0.10238844156265259}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 19:54:02,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676145242489Initializing all the Stores at 1676145242492 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145242493 (+1 ms)Cleaning up temporary data from old regions at 1676145242574 (+81 ms)Cleaning up detritus from prior splits at 1676145242575 (+1 ms)Region opened successfully at 1676145242587 (+12 ms) 2023-02-11 19:54:02,588 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-11 19:54:02,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:02,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:02,615 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:02,617 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-11 19:54:02,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-02-11 19:54:02,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-11 19:54:02,673 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-11 19:54:02,679 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-11 19:54:02,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-11 19:54:02,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-11 19:54:02,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-11 19:54:02,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-11 19:54:02,714 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-11 19:54:02,717 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:02,718 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-11 19:54:02,719 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-11 19:54:02,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-11 19:54:02,740 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 19:54:02,740 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 19:54:02,740 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 19:54:02,740 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 19:54:02,740 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:02,741 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,37191,1676145240670, sessionid=0x10044f292690000, setting cluster-up flag (Was=false) 2023-02-11 19:54:02,757 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:02,762 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-11 19:54:02,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:54:02,769 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:02,775 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-11 19:54:02,776 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:54:02,779 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.hbase-snapshot/.tmp 2023-02-11 19:54:02,816 INFO [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(952): ClusterId : f498648a-9920-464d-aa8f-16d050ee37cb 2023-02-11 19:54:02,816 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(952): ClusterId : f498648a-9920-464d-aa8f-16d050ee37cb 2023-02-11 19:54:02,816 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(952): ClusterId : f498648a-9920-464d-aa8f-16d050ee37cb 2023-02-11 19:54:02,824 DEBUG [RS:2;jenkins-hbase4:42733] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 19:54:02,824 DEBUG [RS:1;jenkins-hbase4:43747] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 19:54:02,824 DEBUG [RS:0;jenkins-hbase4:32837] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 19:54:02,830 DEBUG [RS:1;jenkins-hbase4:43747] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 19:54:02,830 DEBUG [RS:0;jenkins-hbase4:32837] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 19:54:02,830 DEBUG [RS:0;jenkins-hbase4:32837] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 19:54:02,830 DEBUG [RS:2;jenkins-hbase4:42733] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 19:54:02,830 DEBUG [RS:1;jenkins-hbase4:43747] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 19:54:02,830 DEBUG [RS:2;jenkins-hbase4:42733] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 19:54:02,834 DEBUG [RS:0;jenkins-hbase4:32837] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 19:54:02,834 DEBUG [RS:2;jenkins-hbase4:42733] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 19:54:02,836 DEBUG [RS:2;jenkins-hbase4:42733] zookeeper.ReadOnlyZKClient(139): Connect 0x0e0340aa to 127.0.0.1:50506 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 19:54:02,836 DEBUG [RS:0;jenkins-hbase4:32837] zookeeper.ReadOnlyZKClient(139): Connect 0x45cef5bf to 127.0.0.1:50506 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 19:54:02,836 DEBUG [RS:1;jenkins-hbase4:43747] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 19:54:02,838 DEBUG [RS:1;jenkins-hbase4:43747] zookeeper.ReadOnlyZKClient(139): Connect 0x471ef5a4 to 127.0.0.1:50506 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 19:54:02,878 DEBUG [RS:0;jenkins-hbase4:32837] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4b090376, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 19:54:02,879 DEBUG [RS:0;jenkins-hbase4:32837] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4f369ecf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 19:54:02,879 DEBUG [RS:1;jenkins-hbase4:43747] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49d90337, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 19:54:02,879 DEBUG [RS:1;jenkins-hbase4:43747] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4de69ed4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 19:54:02,882 DEBUG [RS:2;jenkins-hbase4:42733] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c4af1a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 19:54:02,882 DEBUG [RS:2;jenkins-hbase4:42733] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4fb2bda, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 19:54:02,904 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:32837 2023-02-11 19:54:02,904 DEBUG [RS:1;jenkins-hbase4:43747] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:43747 2023-02-11 19:54:02,905 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:42733 2023-02-11 19:54:02,908 INFO [RS:0;jenkins-hbase4:32837] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 19:54:02,908 INFO [RS:2;jenkins-hbase4:42733] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 19:54:02,909 INFO [RS:0;jenkins-hbase4:32837] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 19:54:02,908 INFO [RS:1;jenkins-hbase4:43747] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 19:54:02,909 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 19:54:02,909 INFO [RS:2;jenkins-hbase4:42733] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 19:54:02,909 INFO [RS:1;jenkins-hbase4:43747] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 19:54:02,909 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 19:54:02,909 DEBUG [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 19:54:02,911 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37191,1676145240670 with isa=jenkins-hbase4.apache.org/172.31.14.131:32837, startcode=1676145241806 2023-02-11 19:54:02,911 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37191,1676145240670 with isa=jenkins-hbase4.apache.org/172.31.14.131:42733, startcode=1676145241889 2023-02-11 19:54:02,911 INFO [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37191,1676145240670 with isa=jenkins-hbase4.apache.org/172.31.14.131:43747, startcode=1676145241852 2023-02-11 19:54:02,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:02,924 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 19:54:02,924 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 19:54:02,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 19:54:02,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 19:54:02,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-11 19:54:02,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:02,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 19:54:02,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:02,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145272926 2023-02-11 19:54:02,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-11 19:54:02,928 DEBUG [RS:1;jenkins-hbase4:43747] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 19:54:02,928 DEBUG [RS:0;jenkins-hbase4:32837] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 19:54:02,928 DEBUG [RS:2;jenkins-hbase4:42733] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 19:54:02,933 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:02,934 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-11 19:54:02,939 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 19:54:02,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-11 19:54:02,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-11 19:54:02,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-11 19:54:02,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-11 19:54:02,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-11 19:54:02,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:02,950 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-11 19:54:02,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-11 19:54:02,952 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-11 19:54:02,957 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-11 19:54:02,960 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-11 19:54:02,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676145242963,5,FailOnTimeoutGroup] 2023-02-11 19:54:02,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676145242963,5,FailOnTimeoutGroup] 2023-02-11 19:54:02,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:02,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-11 19:54:02,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:02,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:02,989 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:02,991 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:02,991 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f 2023-02-11 19:54:02,995 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52881, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 19:54:02,995 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51823, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 19:54:02,995 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34609, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 19:54:03,017 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37191] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37191] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:03,018 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:03,020 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:03,023 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 19:54:03,025 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/info 2023-02-11 19:54:03,026 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 19:54:03,027 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:03,027 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 19:54:03,031 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/rep_barrier 2023-02-11 19:54:03,031 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 19:54:03,032 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:03,033 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 19:54:03,035 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/table 2023-02-11 19:54:03,036 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 19:54:03,037 DEBUG [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f 2023-02-11 19:54:03,037 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f 2023-02-11 19:54:03,038 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:42039 2023-02-11 19:54:03,037 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f 2023-02-11 19:54:03,038 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 19:54:03,038 DEBUG [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:42039 2023-02-11 19:54:03,038 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:42039 2023-02-11 19:54:03,038 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:03,038 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 19:54:03,038 DEBUG [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 19:54:03,040 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740 2023-02-11 19:54:03,041 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740 2023-02-11 19:54:03,045 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 19:54:03,046 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 19:54:03,048 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 19:54:03,049 DEBUG [RS:0;jenkins-hbase4:32837] zookeeper.ZKUtil(162): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,049 WARN [RS:0;jenkins-hbase4:32837] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 19:54:03,049 INFO [RS:0;jenkins-hbase4:32837] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 19:54:03,050 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1947): logDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,050 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,32837,1676145241806] 2023-02-11 19:54:03,050 DEBUG [RS:1;jenkins-hbase4:43747] zookeeper.ZKUtil(162): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:03,050 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43747,1676145241852] 2023-02-11 19:54:03,051 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42733,1676145241889] 2023-02-11 19:54:03,050 WARN [RS:1;jenkins-hbase4:43747] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 19:54:03,057 INFO [RS:1;jenkins-hbase4:43747] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 19:54:03,056 DEBUG [RS:2;jenkins-hbase4:42733] zookeeper.ZKUtil(162): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:03,057 DEBUG [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1947): logDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:03,057 WARN [RS:2;jenkins-hbase4:42733] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 19:54:03,058 INFO [RS:2;jenkins-hbase4:42733] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 19:54:03,059 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1947): logDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:03,060 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:03,074 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68372669, jitterRate=0.018832162022590637}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 19:54:03,074 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676145243020Initializing all the Stores at 1676145243021 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676145243022 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145243023 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676145243023Cleaning up temporary data from old regions at 1676145243043 (+20 ms)Cleaning up detritus from prior splits at 1676145243044 (+1 ms)Region opened successfully at 1676145243074 (+30 ms) 2023-02-11 19:54:03,075 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 19:54:03,075 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 19:54:03,075 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 19:54:03,075 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 19:54:03,076 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 19:54:03,077 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 19:54:03,077 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676145243075Disabling compacts and flushes for region at 1676145243075Disabling writes for close at 1676145243076 (+1 ms)Writing region close event to WAL at 1676145243077 (+1 ms)Closed at 1676145243077 2023-02-11 19:54:03,081 DEBUG [RS:1;jenkins-hbase4:43747] zookeeper.ZKUtil(162): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,083 DEBUG [RS:1;jenkins-hbase4:43747] zookeeper.ZKUtil(162): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:03,083 DEBUG [RS:1;jenkins-hbase4:43747] zookeeper.ZKUtil(162): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:03,084 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:03,084 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-11 19:54:03,084 DEBUG [RS:0;jenkins-hbase4:32837] zookeeper.ZKUtil(162): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,085 DEBUG [RS:2;jenkins-hbase4:42733] zookeeper.ZKUtil(162): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,085 DEBUG [RS:0;jenkins-hbase4:32837] zookeeper.ZKUtil(162): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:03,085 DEBUG [RS:0;jenkins-hbase4:32837] zookeeper.ZKUtil(162): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:03,085 DEBUG [RS:2;jenkins-hbase4:42733] zookeeper.ZKUtil(162): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:03,087 DEBUG [RS:2;jenkins-hbase4:42733] zookeeper.ZKUtil(162): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:03,095 DEBUG [RS:1;jenkins-hbase4:43747] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 19:54:03,095 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 19:54:03,095 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 19:54:03,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-11 19:54:03,105 INFO [RS:1;jenkins-hbase4:43747] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 19:54:03,105 INFO [RS:2;jenkins-hbase4:42733] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 19:54:03,105 INFO [RS:0;jenkins-hbase4:32837] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 19:54:03,113 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-11 19:54:03,115 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 19:54:03,129 INFO [RS:0;jenkins-hbase4:32837] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 19:54:03,129 INFO [RS:2;jenkins-hbase4:42733] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 19:54:03,129 INFO [RS:1;jenkins-hbase4:43747] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 19:54:03,133 INFO [RS:1;jenkins-hbase4:43747] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 19:54:03,133 INFO [RS:2;jenkins-hbase4:42733] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 19:54:03,134 INFO [RS:0;jenkins-hbase4:32837] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 19:54:03,134 INFO [RS:2;jenkins-hbase4:42733] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,134 INFO [RS:1;jenkins-hbase4:43747] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,134 INFO [RS:0;jenkins-hbase4:32837] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,135 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 19:54:03,135 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 19:54:03,135 INFO [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 19:54:03,143 INFO [RS:2;jenkins-hbase4:42733] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,143 INFO [RS:1;jenkins-hbase4:43747] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,143 INFO [RS:0;jenkins-hbase4:32837] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,143 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 19:54:03,144 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 19:54:03,144 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,145 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,145 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,145 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,145 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,145 DEBUG [RS:2;jenkins-hbase4:42733] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,145 DEBUG [RS:1;jenkins-hbase4:43747] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,144 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,145 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,146 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,146 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,146 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,146 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 19:54:03,146 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,146 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,146 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,146 DEBUG [RS:0;jenkins-hbase4:32837] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 19:54:03,148 INFO [RS:1;jenkins-hbase4:43747] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,148 INFO [RS:2;jenkins-hbase4:42733] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,148 INFO [RS:1;jenkins-hbase4:43747] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,148 INFO [RS:2;jenkins-hbase4:42733] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,148 INFO [RS:1;jenkins-hbase4:43747] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,149 INFO [RS:2;jenkins-hbase4:42733] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,148 INFO [RS:0;jenkins-hbase4:32837] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,149 INFO [RS:0;jenkins-hbase4:32837] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,149 INFO [RS:0;jenkins-hbase4:32837] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,163 INFO [RS:1;jenkins-hbase4:43747] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 19:54:03,165 INFO [RS:1;jenkins-hbase4:43747] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43747,1676145241852-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,165 INFO [RS:0;jenkins-hbase4:32837] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 19:54:03,165 INFO [RS:2;jenkins-hbase4:42733] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 19:54:03,166 INFO [RS:0;jenkins-hbase4:32837] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32837,1676145241806-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,166 INFO [RS:2;jenkins-hbase4:42733] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42733,1676145241889-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,186 INFO [RS:0;jenkins-hbase4:32837] regionserver.Replication(203): jenkins-hbase4.apache.org,32837,1676145241806 started 2023-02-11 19:54:03,187 INFO [RS:2;jenkins-hbase4:42733] regionserver.Replication(203): jenkins-hbase4.apache.org,42733,1676145241889 started 2023-02-11 19:54:03,187 INFO [RS:1;jenkins-hbase4:43747] regionserver.Replication(203): jenkins-hbase4.apache.org,43747,1676145241852 started 2023-02-11 19:54:03,187 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,32837,1676145241806, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:32837, sessionid=0x10044f292690001 2023-02-11 19:54:03,187 INFO [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43747,1676145241852, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43747, sessionid=0x10044f292690002 2023-02-11 19:54:03,187 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42733,1676145241889, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42733, sessionid=0x10044f292690003 2023-02-11 19:54:03,187 DEBUG [RS:1;jenkins-hbase4:43747] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 19:54:03,187 DEBUG [RS:0;jenkins-hbase4:32837] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 19:54:03,187 DEBUG [RS:2;jenkins-hbase4:42733] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 19:54:03,188 DEBUG [RS:2;jenkins-hbase4:42733] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:03,187 DEBUG [RS:0;jenkins-hbase4:32837] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,187 DEBUG [RS:1;jenkins-hbase4:43747] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:03,188 DEBUG [RS:0;jenkins-hbase4:32837] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,32837,1676145241806' 2023-02-11 19:54:03,188 DEBUG [RS:2;jenkins-hbase4:42733] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42733,1676145241889' 2023-02-11 19:54:03,189 DEBUG [RS:0;jenkins-hbase4:32837] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:03,189 DEBUG [RS:1;jenkins-hbase4:43747] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43747,1676145241852' 2023-02-11 19:54:03,189 DEBUG [RS:1;jenkins-hbase4:43747] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:03,189 DEBUG [RS:2;jenkins-hbase4:42733] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:03,190 DEBUG [RS:1;jenkins-hbase4:43747] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:03,190 DEBUG [RS:2;jenkins-hbase4:42733] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:03,190 DEBUG [RS:0;jenkins-hbase4:32837] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:03,190 DEBUG [RS:1;jenkins-hbase4:43747] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 19:54:03,190 DEBUG [RS:1;jenkins-hbase4:43747] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 19:54:03,190 DEBUG [RS:1;jenkins-hbase4:43747] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:03,191 DEBUG [RS:1;jenkins-hbase4:43747] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43747,1676145241852' 2023-02-11 19:54:03,191 DEBUG [RS:1;jenkins-hbase4:43747] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 19:54:03,191 DEBUG [RS:2;jenkins-hbase4:42733] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 19:54:03,191 DEBUG [RS:0;jenkins-hbase4:32837] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 19:54:03,191 DEBUG [RS:2;jenkins-hbase4:42733] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 19:54:03,191 DEBUG [RS:0;jenkins-hbase4:32837] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 19:54:03,191 DEBUG [RS:2;jenkins-hbase4:42733] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:03,191 DEBUG [RS:0;jenkins-hbase4:32837] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,191 DEBUG [RS:2;jenkins-hbase4:42733] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42733,1676145241889' 2023-02-11 19:54:03,192 DEBUG [RS:1;jenkins-hbase4:43747] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 19:54:03,192 DEBUG [RS:2;jenkins-hbase4:42733] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 19:54:03,192 DEBUG [RS:0;jenkins-hbase4:32837] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,32837,1676145241806' 2023-02-11 19:54:03,192 DEBUG [RS:0;jenkins-hbase4:32837] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 19:54:03,192 DEBUG [RS:1;jenkins-hbase4:43747] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 19:54:03,192 INFO [RS:1;jenkins-hbase4:43747] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 19:54:03,193 INFO [RS:1;jenkins-hbase4:43747] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 19:54:03,196 DEBUG [RS:2;jenkins-hbase4:42733] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 19:54:03,196 DEBUG [RS:0;jenkins-hbase4:32837] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 19:54:03,197 DEBUG [RS:2;jenkins-hbase4:42733] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 19:54:03,197 DEBUG [RS:0;jenkins-hbase4:32837] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 19:54:03,197 INFO [RS:2;jenkins-hbase4:42733] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 19:54:03,197 INFO [RS:0;jenkins-hbase4:32837] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 19:54:03,197 INFO [RS:2;jenkins-hbase4:42733] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 19:54:03,198 INFO [RS:0;jenkins-hbase4:32837] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 19:54:03,267 DEBUG [jenkins-hbase4:37191] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-11 19:54:03,271 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:03,277 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:03,277 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:03,277 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:03,277 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:54:03,280 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,32837,1676145241806, state=OPENING 2023-02-11 19:54:03,288 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-11 19:54:03,289 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:03,290 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 19:54:03,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,32837,1676145241806}] 2023-02-11 19:54:03,304 INFO [RS:1;jenkins-hbase4:43747] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43747%2C1676145241852, suffix=, logDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,43747,1676145241852, archiveDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/oldWALs, maxLogs=32 2023-02-11 19:54:03,305 INFO [RS:0;jenkins-hbase4:32837] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C32837%2C1676145241806, suffix=, logDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,32837,1676145241806, archiveDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/oldWALs, maxLogs=32 2023-02-11 19:54:03,305 INFO [RS:2;jenkins-hbase4:42733] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42733%2C1676145241889, suffix=, logDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,42733,1676145241889, archiveDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/oldWALs, maxLogs=32 2023-02-11 19:54:03,343 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK] 2023-02-11 19:54:03,343 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK] 2023-02-11 19:54:03,343 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK] 2023-02-11 19:54:03,344 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK] 2023-02-11 19:54:03,345 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK] 2023-02-11 19:54:03,345 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK] 2023-02-11 19:54:03,358 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK] 2023-02-11 19:54:03,358 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK] 2023-02-11 19:54:03,358 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK] 2023-02-11 19:54:03,360 INFO [RS:2;jenkins-hbase4:42733] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,42733,1676145241889/jenkins-hbase4.apache.org%2C42733%2C1676145241889.1676145243312 2023-02-11 19:54:03,370 DEBUG [RS:2;jenkins-hbase4:42733] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK], DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK], DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK]] 2023-02-11 19:54:03,371 INFO [RS:0;jenkins-hbase4:32837] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,32837,1676145241806/jenkins-hbase4.apache.org%2C32837%2C1676145241806.1676145243312 2023-02-11 19:54:03,371 INFO [RS:1;jenkins-hbase4:43747] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,43747,1676145241852/jenkins-hbase4.apache.org%2C43747%2C1676145241852.1676145243312 2023-02-11 19:54:03,371 DEBUG [RS:0;jenkins-hbase4:32837] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK], DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK], DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK]] 2023-02-11 19:54:03,371 DEBUG [RS:1;jenkins-hbase4:43747] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK], DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK], DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK]] 2023-02-11 19:54:03,481 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:03,484 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 19:54:03,488 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46438, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 19:54:03,503 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-11 19:54:03,503 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 19:54:03,506 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C32837%2C1676145241806.meta, suffix=.meta, logDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,32837,1676145241806, archiveDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/oldWALs, maxLogs=32 2023-02-11 19:54:03,525 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK] 2023-02-11 19:54:03,527 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK] 2023-02-11 19:54:03,527 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK] 2023-02-11 19:54:03,535 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/WALs/jenkins-hbase4.apache.org,32837,1676145241806/jenkins-hbase4.apache.org%2C32837%2C1676145241806.meta.1676145243508.meta 2023-02-11 19:54:03,536 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38329,DS-0daac90e-51bb-4956-8b19-2ed925bb679b,DISK], DatanodeInfoWithStorage[127.0.0.1:44801,DS-3e50c7dd-32c1-4882-95d1-5f3d6789096c,DISK], DatanodeInfoWithStorage[127.0.0.1:37117,DS-58ac90b9-6ded-4f12-a7e7-2cd4ddab6342,DISK]] 2023-02-11 19:54:03,536 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:03,540 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:03,541 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-11 19:54:03,557 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-11 19:54:03,560 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-11 19:54:03,564 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-11 19:54:03,564 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:03,564 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-11 19:54:03,564 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-11 19:54:03,567 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 19:54:03,569 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/info 2023-02-11 19:54:03,569 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/info 2023-02-11 19:54:03,569 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 19:54:03,570 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:03,570 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 19:54:03,572 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/rep_barrier 2023-02-11 19:54:03,572 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/rep_barrier 2023-02-11 19:54:03,572 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 19:54:03,573 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:03,573 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 19:54:03,575 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/table 2023-02-11 19:54:03,575 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/table 2023-02-11 19:54:03,575 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 19:54:03,576 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:03,578 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740 2023-02-11 19:54:03,581 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740 2023-02-11 19:54:03,584 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 19:54:03,586 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 19:54:03,588 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73534756, jitterRate=0.0957532525062561}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 19:54:03,588 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676145243565Writing region info on filesystem at 1676145243565Initializing all the Stores at 1676145243566 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676145243567 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145243567Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676145243567Cleaning up temporary data from old regions at 1676145243583 (+16 ms)Cleaning up detritus from prior splits at 1676145243583Running coprocessor post-open hooks at 1676145243588 (+5 ms)Region opened successfully at 1676145243588 2023-02-11 19:54:03,600 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676145243472 2023-02-11 19:54:03,617 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-11 19:54:03,618 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-11 19:54:03,619 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,32837,1676145241806, state=OPEN 2023-02-11 19:54:03,622 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-11 19:54:03,622 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 19:54:03,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-11 19:54:03,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,32837,1676145241806 in 329 msec 2023-02-11 19:54:03,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-11 19:54:03,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 533 msec 2023-02-11 19:54:03,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 813 msec 2023-02-11 19:54:03,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676145241940, completionTime=-1 2023-02-11 19:54:03,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-11 19:54:03,641 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:03,708 DEBUG [hconnection-0x5876c91-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 19:54:03,711 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46444, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 19:54:03,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:03,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145303727 2023-02-11 19:54:03,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145363727 2023-02-11 19:54:03,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 86 msec 2023-02-11 19:54:03,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37191,1676145240670-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37191,1676145240670-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37191,1676145240670-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37191, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-11 19:54:03,762 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-11 19:54:03,806 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-11 19:54:03,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 19:54:03,818 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:03,821 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:54:03,824 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:54:03,844 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:03,846 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d empty. 2023-02-11 19:54:03,847 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:03,847 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-11 19:54:03,878 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:03,880 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6558cedf10894666d9fab2b1f6f4142d, NAME => 'hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:54:03,899 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:03,899 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 6558cedf10894666d9fab2b1f6f4142d, disabling compactions & flushes 2023-02-11 19:54:03,899 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:54:03,899 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:54:03,899 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. after waiting 0 ms 2023-02-11 19:54:03,899 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:54:03,899 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:54:03,899 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 6558cedf10894666d9fab2b1f6f4142d: Waiting for close lock at 1676145243899Disabling compacts and flushes for region at 1676145243899Disabling writes for close at 1676145243899Writing region close event to WAL at 1676145243899Closed at 1676145243899 2023-02-11 19:54:03,904 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:54:03,919 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676145243906"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145243906"}]},"ts":"1676145243906"} 2023-02-11 19:54:03,944 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:54:03,946 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:54:03,950 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145243946"}]},"ts":"1676145243946"} 2023-02-11 19:54:03,956 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-11 19:54:03,961 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:03,962 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:03,962 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:03,962 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:03,962 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:54:03,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6558cedf10894666d9fab2b1f6f4142d, ASSIGN}] 2023-02-11 19:54:03,968 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6558cedf10894666d9fab2b1f6f4142d, ASSIGN 2023-02-11 19:54:03,969 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6558cedf10894666d9fab2b1f6f4142d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:04,121 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:04,122 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6558cedf10894666d9fab2b1f6f4142d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:04,123 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676145244122"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145244122"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145244122"}]},"ts":"1676145244122"} 2023-02-11 19:54:04,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 6558cedf10894666d9fab2b1f6f4142d, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:04,281 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:04,282 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 19:54:04,286 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45180, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 19:54:04,292 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:54:04,294 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6558cedf10894666d9fab2b1f6f4142d, NAME => 'hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:04,295 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:04,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:04,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:04,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:04,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:04,297 INFO [StoreOpener-6558cedf10894666d9fab2b1f6f4142d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:04,299 DEBUG [StoreOpener-6558cedf10894666d9fab2b1f6f4142d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d/info 2023-02-11 19:54:04,299 DEBUG [StoreOpener-6558cedf10894666d9fab2b1f6f4142d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d/info 2023-02-11 19:54:04,300 INFO [StoreOpener-6558cedf10894666d9fab2b1f6f4142d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6558cedf10894666d9fab2b1f6f4142d columnFamilyName info 2023-02-11 19:54:04,301 INFO [StoreOpener-6558cedf10894666d9fab2b1f6f4142d-1] regionserver.HStore(310): Store=6558cedf10894666d9fab2b1f6f4142d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:04,302 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:04,303 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:04,307 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:54:04,311 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:04,311 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6558cedf10894666d9fab2b1f6f4142d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60547568, jitterRate=-0.09777092933654785}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 19:54:04,312 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6558cedf10894666d9fab2b1f6f4142d: Running coprocessor pre-open hook at 1676145244296Writing region info on filesystem at 1676145244296Initializing all the Stores at 1676145244297 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676145244297Cleaning up temporary data from old regions at 1676145244304 (+7 ms)Cleaning up detritus from prior splits at 1676145244304Running coprocessor post-open hooks at 1676145244312 (+8 ms)Region opened successfully at 1676145244312 2023-02-11 19:54:04,315 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d., pid=6, masterSystemTime=1676145244281 2023-02-11 19:54:04,319 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:54:04,319 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:54:04,321 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:04,321 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676145244320"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145244320"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145244320"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145244320"}]},"ts":"1676145244320"} 2023-02-11 19:54:04,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-11 19:54:04,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 6558cedf10894666d9fab2b1f6f4142d, server=jenkins-hbase4.apache.org,42733,1676145241889 in 197 msec 2023-02-11 19:54:04,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-11 19:54:04,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6558cedf10894666d9fab2b1f6f4142d, ASSIGN in 363 msec 2023-02-11 19:54:04,333 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:54:04,333 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145244333"}]},"ts":"1676145244333"} 2023-02-11 19:54:04,336 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-11 19:54:04,339 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:54:04,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 530 msec 2023-02-11 19:54:04,421 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-11 19:54:04,423 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-11 19:54:04,423 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:04,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 19:54:04,445 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45184, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 19:54:04,458 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:04,474 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 19:54:04,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 28 msec 2023-02-11 19:54:04,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:04,504 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 19:54:04,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-02-11 19:54:04,517 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-11 19:54:04,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-11 19:54:04,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.568sec 2023-02-11 19:54:04,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-11 19:54:04,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-11 19:54:04,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-11 19:54:04,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37191,1676145240670-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-11 19:54:04,527 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37191,1676145240670-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-11 19:54:04,535 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-11 19:54:04,622 DEBUG [Listener at localhost/38771] zookeeper.ReadOnlyZKClient(139): Connect 0x65cede91 to 127.0.0.1:50506 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 19:54:04,628 DEBUG [Listener at localhost/38771] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69e86dd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 19:54:04,641 DEBUG [hconnection-0xe288499-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 19:54:04,652 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46448, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 19:54:04,662 INFO [Listener at localhost/38771] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:54:04,689 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=358, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=275, ProcessCount=168, AvailableMemoryMB=4003 2023-02-11 19:54:04,693 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:04,701 DEBUG [Listener at localhost/38771] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 19:54:04,704 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41904, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 19:54:04,717 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:54:04,717 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:54:04,718 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:54:04,724 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:04,726 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:54:04,728 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:54:04,730 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:04,731 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06 empty. 2023-02-11 19:54:04,733 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:04,733 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-11 19:54:04,756 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:04,757 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 23e1cc59de7125f5225460dcdfdb2b06, NAME => 'testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:54:04,776 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:04,776 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 23e1cc59de7125f5225460dcdfdb2b06, disabling compactions & flushes 2023-02-11 19:54:04,776 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:04,776 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:04,776 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. after waiting 0 ms 2023-02-11 19:54:04,777 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:04,777 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:04,777 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 23e1cc59de7125f5225460dcdfdb2b06: Waiting for close lock at 1676145244776Disabling compacts and flushes for region at 1676145244776Disabling writes for close at 1676145244777 (+1 ms)Writing region close event to WAL at 1676145244777Closed at 1676145244777 2023-02-11 19:54:04,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:54:04,783 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676145244783"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145244783"}]},"ts":"1676145244783"} 2023-02-11 19:54:04,786 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:54:04,788 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:54:04,788 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145244788"}]},"ts":"1676145244788"} 2023-02-11 19:54:04,791 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-11 19:54:04,795 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:04,797 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:04,797 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:04,798 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:04,798 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:54:04,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=23e1cc59de7125f5225460dcdfdb2b06, ASSIGN}] 2023-02-11 19:54:04,801 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=23e1cc59de7125f5225460dcdfdb2b06, ASSIGN 2023-02-11 19:54:04,802 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=23e1cc59de7125f5225460dcdfdb2b06, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:04,952 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:04,953 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=23e1cc59de7125f5225460dcdfdb2b06, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:04,954 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676145244953"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145244953"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145244953"}]},"ts":"1676145244953"} 2023-02-11 19:54:04,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 23e1cc59de7125f5225460dcdfdb2b06, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:05,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 23e1cc59de7125f5225460dcdfdb2b06, NAME => 'testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:05,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:05,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:05,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:05,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:05,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:05,120 INFO [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:05,122 DEBUG [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1 2023-02-11 19:54:05,122 DEBUG [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1 2023-02-11 19:54:05,122 INFO [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 23e1cc59de7125f5225460dcdfdb2b06 columnFamilyName cf1 2023-02-11 19:54:05,123 INFO [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] regionserver.HStore(310): Store=23e1cc59de7125f5225460dcdfdb2b06/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:05,123 INFO [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:05,125 DEBUG [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2 2023-02-11 19:54:05,125 DEBUG [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2 2023-02-11 19:54:05,126 INFO [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 23e1cc59de7125f5225460dcdfdb2b06 columnFamilyName cf2 2023-02-11 19:54:05,126 INFO [StoreOpener-23e1cc59de7125f5225460dcdfdb2b06-1] regionserver.HStore(310): Store=23e1cc59de7125f5225460dcdfdb2b06/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:05,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:05,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:05,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:05,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:05,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:05,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 23e1cc59de7125f5225460dcdfdb2b06; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65907745, jitterRate=-0.017898067831993103}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:05,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 23e1cc59de7125f5225460dcdfdb2b06: Running coprocessor pre-open hook at 1676145245118Writing region info on filesystem at 1676145245118Initializing all the Stores at 1676145245119 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145245120 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145245120Cleaning up temporary data from old regions at 1676145245130 (+10 ms)Cleaning up detritus from prior splits at 1676145245130Running coprocessor post-open hooks at 1676145245138 (+8 ms)Region opened successfully at 1676145245138 2023-02-11 19:54:05,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06., pid=11, masterSystemTime=1676145245110 2023-02-11 19:54:05,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,143 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=23e1cc59de7125f5225460dcdfdb2b06, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,144 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676145245143"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145245143"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145245143"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145245143"}]},"ts":"1676145245143"} 2023-02-11 19:54:05,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-11 19:54:05,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 23e1cc59de7125f5225460dcdfdb2b06, server=jenkins-hbase4.apache.org,42733,1676145241889 in 190 msec 2023-02-11 19:54:05,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-11 19:54:05,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=23e1cc59de7125f5225460dcdfdb2b06, ASSIGN in 352 msec 2023-02-11 19:54:05,155 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:54:05,155 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145245155"}]},"ts":"1676145245155"} 2023-02-11 19:54:05,157 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-11 19:54:05,160 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:54:05,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 438 msec 2023-02-11 19:54:05,229 DEBUG [Listener at localhost/38771] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 19:54:05,233 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45194, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 19:54:05,251 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:05,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,259 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:05,259 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,260 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 19:54:05,260 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 19:54:05,260 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,260 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,262 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,262 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,262 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,262 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,262 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,262 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,266 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,266 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,266 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,268 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,269 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:05,269 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:05,269 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,269 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. started... 2023-02-11 19:54:05,270 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 23e1cc59de7125f5225460dcdfdb2b06 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 19:54:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,341 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/6f2473a6ea6b46138b82b2974897289b 2023-02-11 19:54:05,407 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/7a99efe5ca7b4d5da2db8038ad247ddf 2023-02-11 19:54:05,417 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/6f2473a6ea6b46138b82b2974897289b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b 2023-02-11 19:54:05,426 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:54:05,428 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/7a99efe5ca7b4d5da2db8038ad247ddf as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf 2023-02-11 19:54:05,436 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:54:05,438 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 23e1cc59de7125f5225460dcdfdb2b06 in 168ms, sequenceid=6, compaction requested=false 2023-02-11 19:54:05,440 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:05,442 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 23e1cc59de7125f5225460dcdfdb2b06: 2023-02-11 19:54:05,442 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,446 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,447 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:05,447 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,448 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:05,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 19:54:05,451 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:05,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32794b40[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:05,451 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,455 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,455 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,455 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,456 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,456 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:05,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,457 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,457 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,457 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:05,460 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,461 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,461 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,461 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,461 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,461 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,461 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,462 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,462 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:05,462 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:05,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:05,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:05,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:05,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:05,468 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 19:54:05,468 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:05,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:05,468 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:05,468 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,468 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:05,470 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,470 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,470 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,470 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,470 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,470 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:05,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:05,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,471 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,471 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:05,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,471 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,472 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,475 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:05,475 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,475 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,487 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,487 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,487 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:05,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,487 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,487 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:05,487 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,488 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,488 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,488 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,488 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,488 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,489 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,489 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,489 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,489 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,490 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:05,490 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:05,590 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:05,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:05,619 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:05,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:05,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,623 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:05,623 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,624 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 19:54:05,624 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 19:54:05,624 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,624 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,627 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,627 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,627 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,627 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,627 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,628 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:05,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,629 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:05,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,629 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,629 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,630 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,630 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:05,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:05,630 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,630 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,630 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,630 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,630 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,631 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,632 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,632 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,632 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,632 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:05,632 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:05,632 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,632 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. started... 2023-02-11 19:54:05,633 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 23e1cc59de7125f5225460dcdfdb2b06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:05,634 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,635 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,635 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,635 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,635 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,635 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,656 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/de47f19d894d43bbb2e5b54456cdfcaf 2023-02-11 19:54:05,681 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/b808ed1892d04ad49703d51807c592f8 2023-02-11 19:54:05,690 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/de47f19d894d43bbb2e5b54456cdfcaf as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf 2023-02-11 19:54:05,698 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:54:05,699 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/b808ed1892d04ad49703d51807c592f8 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8 2023-02-11 19:54:05,706 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:54:05,708 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 23e1cc59de7125f5225460dcdfdb2b06 in 76ms, sequenceid=19, compaction requested=false 2023-02-11 19:54:05,708 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 23e1cc59de7125f5225460dcdfdb2b06: 2023-02-11 19:54:05,708 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,708 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:05,709 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:05,709 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,709 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,709 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,711 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,711 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:05,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:05,711 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,711 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:05,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 19:54:05,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1cce465d[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:05,714 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:05,714 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,716 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:05,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:05,716 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,717 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:05,719 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,719 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,719 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,720 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,720 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:05,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:05,720 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,720 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,721 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,721 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:05,721 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:05,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:05,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:05,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:05,726 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 19:54:05,726 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:05,726 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:05,726 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,726 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:05,729 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,729 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,729 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,729 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,729 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,730 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:05,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:05,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,730 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,730 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,731 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:05,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,731 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:05,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,734 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:05,735 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,736 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,746 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:05,746 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,746 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,747 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:05,747 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:05,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:05,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:05,847 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:05,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:05,872 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:05,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:05,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,874 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:05,874 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,875 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 19:54:05,875 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 19:54:05,876 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,876 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,878 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,878 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,878 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,878 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:05,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:05,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:05,878 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,879 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:05,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:05,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,880 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,881 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,881 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,881 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,881 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,881 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:05,881 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,881 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:05,882 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,882 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:05,882 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,882 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,882 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:05,882 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:05,882 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,883 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. started... 2023-02-11 19:54:05,883 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 23e1cc59de7125f5225460dcdfdb2b06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:05,884 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,885 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,885 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,885 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,886 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,886 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,919 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/1e7751d340aa4cd7abb5a88e1f769996 2023-02-11 19:54:05,954 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/e2dd487614e343469ce250d0764fb4ea 2023-02-11 19:54:05,962 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/1e7751d340aa4cd7abb5a88e1f769996 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996 2023-02-11 19:54:05,970 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:54:05,971 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/e2dd487614e343469ce250d0764fb4ea as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea 2023-02-11 19:54:05,979 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:54:05,980 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 23e1cc59de7125f5225460dcdfdb2b06 in 97ms, sequenceid=32, compaction requested=true 2023-02-11 19:54:05,980 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 23e1cc59de7125f5225460dcdfdb2b06: 2023-02-11 19:54:05,980 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:05,980 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:05,981 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:05,981 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,981 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:05,981 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,985 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:05,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 19:54:05,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a2dc784[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:05,987 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:05,987 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,989 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,989 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:05,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:05,992 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,992 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,992 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,993 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,993 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,993 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:05,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:05,993 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,995 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:05,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:05,995 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:05,995 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:05,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:05,999 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 19:54:05,999 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:05,999 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:05,999 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:05,999 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:06,001 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,001 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,001 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,002 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,001 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,002 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,006 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,006 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,007 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,010 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,010 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,010 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,010 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,011 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,010 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,010 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,011 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:06,012 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:06,012 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,011 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,012 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,012 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:06,012 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,013 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:06,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,113 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:06,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:06,137 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:06,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:06,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,140 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:06,140 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,141 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 19:54:06,141 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 19:54:06,141 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,141 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,143 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,143 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,143 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,143 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,143 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,143 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,145 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,145 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,146 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,146 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,146 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,146 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,146 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:06,146 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,146 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:06,146 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,147 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,146 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,146 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:06,147 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,147 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,147 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. started... 2023-02-11 19:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,148 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 23e1cc59de7125f5225460dcdfdb2b06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,152 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,153 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,153 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,153 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,153 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,153 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,180 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/57c47c5fdf4448a3bd13a506d60130a1 2023-02-11 19:54:06,211 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/676e8ab1791c432faacef36fb7c06082 2023-02-11 19:54:06,228 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/57c47c5fdf4448a3bd13a506d60130a1 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1 2023-02-11 19:54:06,239 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:54:06,241 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/676e8ab1791c432faacef36fb7c06082 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082 2023-02-11 19:54:06,249 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:54:06,250 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 23e1cc59de7125f5225460dcdfdb2b06 in 102ms, sequenceid=45, compaction requested=true 2023-02-11 19:54:06,250 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 23e1cc59de7125f5225460dcdfdb2b06: 2023-02-11 19:54:06,250 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:06,250 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:06,251 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:06,251 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,251 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,251 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,255 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,256 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,256 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,256 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 19:54:06,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76026c94[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:06,259 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:06,259 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,261 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,261 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,261 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,262 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,262 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:06,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:06,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:06,263 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,263 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,263 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,263 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,264 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,265 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,265 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,265 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,265 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,266 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:06,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:06,271 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 19:54:06,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:06,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:06,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,271 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:06,273 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,273 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,273 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,274 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,273 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,274 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,275 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,277 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,277 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,277 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,284 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,284 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,284 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,284 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,284 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,285 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,284 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,285 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:06,285 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:06,285 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,285 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,284 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,286 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,285 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,286 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:06,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,286 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,286 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:06,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,286 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,386 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:06,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:06,409 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:06,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:06,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,412 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:06,412 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,412 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 19:54:06,413 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 19:54:06,413 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,413 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,414 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,414 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,414 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,414 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,415 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,415 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:06,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,415 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,416 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,417 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,417 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,417 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,417 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,417 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,417 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,417 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,417 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:06,422 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:06,422 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. started... 2023-02-11 19:54:06,422 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 23e1cc59de7125f5225460dcdfdb2b06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:06,423 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,424 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,424 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,424 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,424 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,424 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,441 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/0d256eba79fe4a709594cfe8007c659a 2023-02-11 19:54:06,468 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/2adc5041e32847cd972066f0501b36cc 2023-02-11 19:54:06,476 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/0d256eba79fe4a709594cfe8007c659a as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/0d256eba79fe4a709594cfe8007c659a 2023-02-11 19:54:06,483 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/0d256eba79fe4a709594cfe8007c659a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:54:06,484 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/2adc5041e32847cd972066f0501b36cc as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/2adc5041e32847cd972066f0501b36cc 2023-02-11 19:54:06,491 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/2adc5041e32847cd972066f0501b36cc, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:54:06,492 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 23e1cc59de7125f5225460dcdfdb2b06 in 70ms, sequenceid=58, compaction requested=true 2023-02-11 19:54:06,493 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 23e1cc59de7125f5225460dcdfdb2b06: 2023-02-11 19:54:06,493 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,494 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,494 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,496 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,496 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,497 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,497 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 19:54:06,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e590a5f[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:06,499 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:06,499 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,501 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:06,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,500 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,501 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,501 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,501 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,501 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,501 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:06,501 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:06,501 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,501 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:06,502 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,502 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,502 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,503 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,502 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,503 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,503 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,504 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,504 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,504 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:06,510 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:06,510 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 19:54:06,510 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:06,510 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:06,510 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,510 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:06,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,513 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,513 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,515 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,516 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,516 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,522 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,522 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:06,522 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:06,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:06,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,524 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,623 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:06,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:06,645 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:06,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:06,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,647 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:06,648 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,648 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 19:54:06,648 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 19:54:06,649 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,649 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,650 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,650 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,650 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,650 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,651 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,651 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:06,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,652 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,653 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 19:54:06,653 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,653 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,653 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,653 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,653 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,654 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:06,654 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:06,653 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,654 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 19:54:06,654 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,654 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:06,654 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,655 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. started... 2023-02-11 19:54:06,655 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,654 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:06,655 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 19:54:06,655 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,655 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,656 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 23e1cc59de7125f5225460dcdfdb2b06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:06,657 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,659 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,659 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,659 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,659 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,659 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,676 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/077f7aa808be4a158f724a5924d94125 2023-02-11 19:54:06,696 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/f984e4e0508b431099798dce68b16e5f 2023-02-11 19:54:06,703 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/077f7aa808be4a158f724a5924d94125 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/077f7aa808be4a158f724a5924d94125 2023-02-11 19:54:06,711 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/077f7aa808be4a158f724a5924d94125, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:54:06,712 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/f984e4e0508b431099798dce68b16e5f as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/f984e4e0508b431099798dce68b16e5f 2023-02-11 19:54:06,718 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/f984e4e0508b431099798dce68b16e5f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:54:06,720 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 23e1cc59de7125f5225460dcdfdb2b06 in 64ms, sequenceid=71, compaction requested=true 2023-02-11 19:54:06,720 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 23e1cc59de7125f5225460dcdfdb2b06: 2023-02-11 19:54:06,720 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:06,720 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 19:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 19:54:06,723 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,723 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,723 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,723 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:06,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 19:54:06,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@604b5967[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:06,725 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:06,725 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,726 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,726 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,726 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,727 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,727 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:06,727 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:06,729 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,729 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,729 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,729 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,731 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,731 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 19:54:06,731 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,731 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,731 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,732 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 19:54:06,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:06,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:06,735 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 19:54:06,735 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:06,735 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:06,735 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,735 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:06,736 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,736 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,736 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,736 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,736 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,737 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:06,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:06,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,736 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,737 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:06,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,737 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,738 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:06,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,740 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:06,740 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,740 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,744 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:06,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,744 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:06,744 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 19:54:06,745 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 19:54:06,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:06,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,745 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:06,746 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:06,846 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:06,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:06,874 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,882 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:06,883 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:06,883 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:06,883 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:06,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=23e1cc59de7125f5225460dcdfdb2b06, UNASSIGN}] 2023-02-11 19:54:06,895 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=23e1cc59de7125f5225460dcdfdb2b06, UNASSIGN 2023-02-11 19:54:06,896 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=23e1cc59de7125f5225460dcdfdb2b06, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:06,896 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676145246896"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145246896"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145246896"}]},"ts":"1676145246896"} 2023-02-11 19:54:06,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 23e1cc59de7125f5225460dcdfdb2b06, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:07,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 23e1cc59de7125f5225460dcdfdb2b06, disabling compactions & flushes 2023-02-11 19:54:07,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:07,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:07,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. after waiting 0 ms 2023-02-11 19:54:07,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:07,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 23e1cc59de7125f5225460dcdfdb2b06 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 19:54:07,083 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/e35e7083e26647d8b726dd1e6249a362 2023-02-11 19:54:07,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/7a4c27fe15974ec38a0ac636b491797b 2023-02-11 19:54:07,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf1/e35e7083e26647d8b726dd1e6249a362 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/e35e7083e26647d8b726dd1e6249a362 2023-02-11 19:54:07,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/e35e7083e26647d8b726dd1e6249a362, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:54:07,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.tmp/cf2/7a4c27fe15974ec38a0ac636b491797b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a4c27fe15974ec38a0ac636b491797b 2023-02-11 19:54:07,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a4c27fe15974ec38a0ac636b491797b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:54:07,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 23e1cc59de7125f5225460dcdfdb2b06 in 77ms, sequenceid=82, compaction requested=true 2023-02-11 19:54:07,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 19:54:07,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:07,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. 2023-02-11 19:54:07,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 23e1cc59de7125f5225460dcdfdb2b06: Waiting for close lock at 1676145247061Running coprocessor pre-close hooks at 1676145247061Disabling compacts and flushes for region at 1676145247061Disabling writes for close at 1676145247061Obtaining lock to block concurrent updates at 1676145247061Preparing flush snapshotting stores in 23e1cc59de7125f5225460dcdfdb2b06 at 1676145247061Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676145247062 (+1 ms)Flushing stores of testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. at 1676145247062Flushing 23e1cc59de7125f5225460dcdfdb2b06/cf1: creating writer at 1676145247063 (+1 ms)Flushing 23e1cc59de7125f5225460dcdfdb2b06/cf1: appending metadata at 1676145247066 (+3 ms)Flushing 23e1cc59de7125f5225460dcdfdb2b06/cf1: closing flushed file at 1676145247067 (+1 ms)Flushing 23e1cc59de7125f5225460dcdfdb2b06/cf2: creating writer at 1676145247090 (+23 ms)Flushing 23e1cc59de7125f5225460dcdfdb2b06/cf2: appending metadata at 1676145247093 (+3 ms)Flushing 23e1cc59de7125f5225460dcdfdb2b06/cf2: closing flushed file at 1676145247093Flushing 23e1cc59de7125f5225460dcdfdb2b06/cf1: reopening flushed file at 1676145247120 (+27 ms)Flushing 23e1cc59de7125f5225460dcdfdb2b06/cf2: reopening flushed file at 1676145247131 (+11 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 23e1cc59de7125f5225460dcdfdb2b06 in 77ms, sequenceid=82, compaction requested=true at 1676145247138 (+7 ms)Writing region close event to WAL at 1676145247155 (+17 ms)Running coprocessor post-close hooks at 1676145247161 (+6 ms)Closed at 1676145247161 2023-02-11 19:54:07,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,165 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=23e1cc59de7125f5225460dcdfdb2b06, regionState=CLOSED 2023-02-11 19:54:07,166 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676145247165"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145247165"}]},"ts":"1676145247165"} 2023-02-11 19:54:07,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-11 19:54:07,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 23e1cc59de7125f5225460dcdfdb2b06, server=jenkins-hbase4.apache.org,42733,1676145241889 in 269 msec 2023-02-11 19:54:07,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-11 19:54:07,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=23e1cc59de7125f5225460dcdfdb2b06, UNASSIGN in 277 msec 2023-02-11 19:54:07,192 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=23e1cc59de7125f5225460dcdfdb2b06, threads=14 2023-02-11 19:54:07,193 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/2adc5041e32847cd972066f0501b36cc for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,193 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,193 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a4c27fe15974ec38a0ac636b491797b for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,193 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,194 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,194 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,194 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/f984e4e0508b431099798dce68b16e5f for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,194 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/077f7aa808be4a158f724a5924d94125 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,194 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/0d256eba79fe4a709594cfe8007c659a for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,195 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,195 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,195 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,195 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,196 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/e35e7083e26647d8b726dd1e6249a362 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,223 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/2adc5041e32847cd972066f0501b36cc, top=true 2023-02-11 19:54:07,244 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/f984e4e0508b431099798dce68b16e5f, top=true 2023-02-11 19:54:07,278 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/e35e7083e26647d8b726dd1e6249a362, top=true 2023-02-11 19:54:07,279 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/077f7aa808be4a158f724a5924d94125, top=true 2023-02-11 19:54:07,282 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a4c27fe15974ec38a0ac636b491797b, top=true 2023-02-11 19:54:07,294 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/0d256eba79fe4a709594cfe8007c659a, top=true 2023-02-11 19:54:07,299 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.splits/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-f984e4e0508b431099798dce68b16e5f for child: 53477071b15ad5d16d882e1728ca27a7, parent: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,299 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.splits/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-2adc5041e32847cd972066f0501b36cc for child: 53477071b15ad5d16d882e1728ca27a7, parent: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,299 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/f984e4e0508b431099798dce68b16e5f for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,299 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/2adc5041e32847cd972066f0501b36cc for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,340 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.splits/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-0d256eba79fe4a709594cfe8007c659a for child: 53477071b15ad5d16d882e1728ca27a7, parent: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,340 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/0d256eba79fe4a709594cfe8007c659a for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,341 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.splits/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-7a4c27fe15974ec38a0ac636b491797b for child: 53477071b15ad5d16d882e1728ca27a7, parent: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,341 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.splits/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-077f7aa808be4a158f724a5924d94125 for child: 53477071b15ad5d16d882e1728ca27a7, parent: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,341 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a4c27fe15974ec38a0ac636b491797b for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,341 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/077f7aa808be4a158f724a5924d94125 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,344 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/.splits/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-e35e7083e26647d8b726dd1e6249a362 for child: 53477071b15ad5d16d882e1728ca27a7, parent: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,344 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/e35e7083e26647d8b726dd1e6249a362 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,395 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,396 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,403 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,413 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,416 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,418 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996 for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,419 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,794 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b for region: 23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:07,795 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 23e1cc59de7125f5225460dcdfdb2b06 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:54:07,910 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:54:07,912 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:54:07,915 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676145247915"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676145247915"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676145247915"}]},"ts":"1676145247915"} 2023-02-11 19:54:07,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145247915"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145247915"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145247915"}]},"ts":"1676145247915"} 2023-02-11 19:54:07,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145247915"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145247915"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145247915"}]},"ts":"1676145247915"} 2023-02-11 19:54:07,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=740882a55424963b0009dd2ab104ea35, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=53477071b15ad5d16d882e1728ca27a7, ASSIGN}] 2023-02-11 19:54:07,970 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=53477071b15ad5d16d882e1728ca27a7, ASSIGN 2023-02-11 19:54:07,971 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=53477071b15ad5d16d882e1728ca27a7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:07,972 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=740882a55424963b0009dd2ab104ea35, ASSIGN 2023-02-11 19:54:07,973 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=740882a55424963b0009dd2ab104ea35, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:08,121 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 19:54:08,123 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=53477071b15ad5d16d882e1728ca27a7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:08,123 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145248122"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145248122"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145248122"}]},"ts":"1676145248122"} 2023-02-11 19:54:08,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 53477071b15ad5d16d882e1728ca27a7, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:08,129 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=740882a55424963b0009dd2ab104ea35, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:08,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145248129"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145248129"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145248129"}]},"ts":"1676145248129"} 2023-02-11 19:54:08,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 740882a55424963b0009dd2ab104ea35, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:08,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:08,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 53477071b15ad5d16d882e1728ca27a7, NAME => 'testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.', STARTKEY => '26', ENDKEY => ''} 2023-02-11 19:54:08,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:08,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:08,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:08,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:08,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:08,285 INFO [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:08,287 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1 2023-02-11 19:54:08,287 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1 2023-02-11 19:54:08,288 INFO [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53477071b15ad5d16d882e1728ca27a7 columnFamilyName cf1 2023-02-11 19:54:08,329 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996-top 2023-02-11 19:54:08,337 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1-top 2023-02-11 19:54:08,347 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b-top 2023-02-11 19:54:08,359 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf-top 2023-02-11 19:54:08,368 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-077f7aa808be4a158f724a5924d94125 2023-02-11 19:54:08,374 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-0d256eba79fe4a709594cfe8007c659a 2023-02-11 19:54:08,380 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-e35e7083e26647d8b726dd1e6249a362 2023-02-11 19:54:08,381 INFO [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(310): Store=53477071b15ad5d16d882e1728ca27a7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:08,381 INFO [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:08,382 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2 2023-02-11 19:54:08,382 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2 2023-02-11 19:54:08,383 INFO [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53477071b15ad5d16d882e1728ca27a7 columnFamilyName cf2 2023-02-11 19:54:08,397 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082-top 2023-02-11 19:54:08,402 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf-top 2023-02-11 19:54:08,408 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8-top 2023-02-11 19:54:08,424 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea-top 2023-02-11 19:54:08,432 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-2adc5041e32847cd972066f0501b36cc 2023-02-11 19:54:08,440 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-7a4c27fe15974ec38a0ac636b491797b 2023-02-11 19:54:08,447 DEBUG [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-f984e4e0508b431099798dce68b16e5f 2023-02-11 19:54:08,447 INFO [StoreOpener-53477071b15ad5d16d882e1728ca27a7-1] regionserver.HStore(310): Store=53477071b15ad5d16d882e1728ca27a7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:08,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:08,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:08,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:08,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:08,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 53477071b15ad5d16d882e1728ca27a7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60085739, jitterRate=-0.10465271770954132}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:08,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 53477071b15ad5d16d882e1728ca27a7: Running coprocessor pre-open hook at 1676145248284Writing region info on filesystem at 1676145248284Initializing all the Stores at 1676145248285 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145248285Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145248285Cleaning up temporary data from old regions at 1676145248452 (+167 ms)Cleaning up detritus from prior splits at 1676145248452Running coprocessor post-open hooks at 1676145248457 (+5 ms)Region opened successfully at 1676145248457 2023-02-11 19:54:08,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7., pid=17, masterSystemTime=1676145248277 2023-02-11 19:54:08,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:08,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:54:08,463 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:08,463 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:08,466 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:08,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:08,466 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): 53477071b15ad5d16d882e1728ca27a7/cf2 is initiating minor compaction (all files) 2023-02-11 19:54:08,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:08,467 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 53477071b15ad5d16d882e1728ca27a7/cf2 in testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:08,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:08,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 740882a55424963b0009dd2ab104ea35, NAME => 'testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.', STARTKEY => '', ENDKEY => '26'} 2023-02-11 19:54:08,467 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-2adc5041e32847cd972066f0501b36cc, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-f984e4e0508b431099798dce68b16e5f, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-7a4c27fe15974ec38a0ac636b491797b] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/.tmp, totalSize=34.9 K 2023-02-11 19:54:08,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:08,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:08,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:08,468 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:08,469 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): 53477071b15ad5d16d882e1728ca27a7/cf1 is initiating minor compaction (all files) 2023-02-11 19:54:08,469 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 53477071b15ad5d16d882e1728ca27a7/cf1 in testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:08,470 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=53477071b15ad5d16d882e1728ca27a7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:08,470 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145248469"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145248469"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145248469"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145248469"}]},"ts":"1676145248469"} 2023-02-11 19:54:08,470 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-0d256eba79fe4a709594cfe8007c659a, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-077f7aa808be4a158f724a5924d94125, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-e35e7083e26647d8b726dd1e6249a362] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/.tmp, totalSize=34.9 K 2023-02-11 19:54:08,470 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145245234 2023-02-11 19:54:08,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:08,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:08,475 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145245597 2023-02-11 19:54:08,475 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145245234 2023-02-11 19:54:08,476 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145245850 2023-02-11 19:54:08,479 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676145246116 2023-02-11 19:54:08,479 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145245597 2023-02-11 19:54:08,479 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-2adc5041e32847cd972066f0501b36cc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145246389 2023-02-11 19:54:08,480 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145245850 2023-02-11 19:54:08,481 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-f984e4e0508b431099798dce68b16e5f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145246625 2023-02-11 19:54:08,483 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-7a4c27fe15974ec38a0ac636b491797b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145246851 2023-02-11 19:54:08,483 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676145246116 2023-02-11 19:54:08,484 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-0d256eba79fe4a709594cfe8007c659a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145246389 2023-02-11 19:54:08,486 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-077f7aa808be4a158f724a5924d94125, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145246625 2023-02-11 19:54:08,487 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-e35e7083e26647d8b726dd1e6249a362, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145246851 2023-02-11 19:54:08,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-11 19:54:08,489 INFO [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:08,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 53477071b15ad5d16d882e1728ca27a7, server=jenkins-hbase4.apache.org,42733,1676145241889 in 351 msec 2023-02-11 19:54:08,491 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1 2023-02-11 19:54:08,491 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1 2023-02-11 19:54:08,492 INFO [StoreOpener-740882a55424963b0009dd2ab104ea35-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 740882a55424963b0009dd2ab104ea35 columnFamilyName cf1 2023-02-11 19:54:08,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=53477071b15ad5d16d882e1728ca27a7, ASSIGN in 521 msec 2023-02-11 19:54:08,548 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): 53477071b15ad5d16d882e1728ca27a7#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:08,555 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996-bottom 2023-02-11 19:54:08,562 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1-bottom 2023-02-11 19:54:08,568 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b-bottom 2023-02-11 19:54:08,579 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf-bottom 2023-02-11 19:54:08,594 INFO [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(310): Store=740882a55424963b0009dd2ab104ea35/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:08,594 INFO [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:08,597 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): 53477071b15ad5d16d882e1728ca27a7#cf1#compaction#15 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:08,611 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2 2023-02-11 19:54:08,611 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2 2023-02-11 19:54:08,611 INFO [StoreOpener-740882a55424963b0009dd2ab104ea35-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 740882a55424963b0009dd2ab104ea35 columnFamilyName cf2 2023-02-11 19:54:08,641 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/.tmp/cf2/dce7bebacae94b82b39484403edac6ab as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/dce7bebacae94b82b39484403edac6ab 2023-02-11 19:54:08,660 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 53477071b15ad5d16d882e1728ca27a7/cf2 of 53477071b15ad5d16d882e1728ca27a7 into dce7bebacae94b82b39484403edac6ab(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:08,661 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 53477071b15ad5d16d882e1728ca27a7: 2023-02-11 19:54:08,661 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7., storeName=53477071b15ad5d16d882e1728ca27a7/cf2, priority=9, startTime=1676145248462; duration=0sec 2023-02-11 19:54:08,661 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082-bottom 2023-02-11 19:54:08,661 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:08,669 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf-bottom 2023-02-11 19:54:08,678 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8-bottom 2023-02-11 19:54:08,679 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/.tmp/cf1/ffe33da3dd8e43e2ab09d58982d69004 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/ffe33da3dd8e43e2ab09d58982d69004 2023-02-11 19:54:08,684 DEBUG [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea-bottom 2023-02-11 19:54:08,685 INFO [StoreOpener-740882a55424963b0009dd2ab104ea35-1] regionserver.HStore(310): Store=740882a55424963b0009dd2ab104ea35/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:08,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:08,688 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 53477071b15ad5d16d882e1728ca27a7/cf1 of 53477071b15ad5d16d882e1728ca27a7 into ffe33da3dd8e43e2ab09d58982d69004(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:08,689 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 53477071b15ad5d16d882e1728ca27a7: 2023-02-11 19:54:08,689 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7., storeName=53477071b15ad5d16d882e1728ca27a7/cf1, priority=9, startTime=1676145248461; duration=0sec 2023-02-11 19:54:08,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:08,689 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:08,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:08,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:08,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 740882a55424963b0009dd2ab104ea35; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68584856, jitterRate=0.02199399471282959}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:08,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 740882a55424963b0009dd2ab104ea35: Running coprocessor pre-open hook at 1676145248474Writing region info on filesystem at 1676145248474Initializing all the Stores at 1676145248480 (+6 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145248481 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145248481Cleaning up temporary data from old regions at 1676145248690 (+209 ms)Cleaning up detritus from prior splits at 1676145248690Running coprocessor post-open hooks at 1676145248694 (+4 ms)Region opened successfully at 1676145248694 2023-02-11 19:54:08,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35., pid=18, masterSystemTime=1676145248277 2023-02-11 19:54:08,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:08,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:54:08,695 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 19:54:08,695 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 19:54:08,697 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:08,698 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:08,698 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): 740882a55424963b0009dd2ab104ea35/cf1 is initiating minor compaction (all files) 2023-02-11 19:54:08,698 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): 740882a55424963b0009dd2ab104ea35/cf2 is initiating minor compaction (all files) 2023-02-11 19:54:08,698 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 740882a55424963b0009dd2ab104ea35/cf1 in testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:08,698 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 740882a55424963b0009dd2ab104ea35/cf2 in testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:08,698 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/.tmp, totalSize=19.9 K 2023-02-11 19:54:08,698 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/.tmp, totalSize=19.9 K 2023-02-11 19:54:08,700 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145245234 2023-02-11 19:54:08,700 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145245234 2023-02-11 19:54:08,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:08,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:08,701 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145245597 2023-02-11 19:54:08,701 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145245597 2023-02-11 19:54:08,702 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145245850 2023-02-11 19:54:08,702 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=740882a55424963b0009dd2ab104ea35, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:08,702 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145248702"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145248702"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145248702"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145248702"}]},"ts":"1676145248702"} 2023-02-11 19:54:08,702 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145246116 2023-02-11 19:54:08,703 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145245850 2023-02-11 19:54:08,704 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145246116 2023-02-11 19:54:08,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-11 19:54:08,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 740882a55424963b0009dd2ab104ea35, server=jenkins-hbase4.apache.org,42733,1676145241889 in 574 msec 2023-02-11 19:54:08,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-11 19:54:08,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=740882a55424963b0009dd2ab104ea35, ASSIGN in 743 msec 2023-02-11 19:54:08,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 in 1.8410 sec 2023-02-11 19:54:08,721 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): 740882a55424963b0009dd2ab104ea35#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:08,721 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): 740882a55424963b0009dd2ab104ea35#cf2#compaction#16 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:08,735 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/.tmp/cf2/dc37cd42d3f3446da0d7aac190381df3 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/dc37cd42d3f3446da0d7aac190381df3 2023-02-11 19:54:08,735 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/.tmp/cf1/7e8ab368bb8e4b7bbf33deadf34efe65 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/7e8ab368bb8e4b7bbf33deadf34efe65 2023-02-11 19:54:08,741 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 740882a55424963b0009dd2ab104ea35/cf2 of 740882a55424963b0009dd2ab104ea35 into dc37cd42d3f3446da0d7aac190381df3(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:08,741 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 740882a55424963b0009dd2ab104ea35/cf1 of 740882a55424963b0009dd2ab104ea35 into 7e8ab368bb8e4b7bbf33deadf34efe65(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:08,741 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 740882a55424963b0009dd2ab104ea35: 2023-02-11 19:54:08,741 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 740882a55424963b0009dd2ab104ea35: 2023-02-11 19:54:08,741 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35., storeName=740882a55424963b0009dd2ab104ea35/cf2, priority=12, startTime=1676145248695; duration=0sec 2023-02-11 19:54:08,741 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35., storeName=740882a55424963b0009dd2ab104ea35/cf1, priority=12, startTime=1676145248695; duration=0sec 2023-02-11 19:54:08,741 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:08,741 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:08,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42733] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:45194 deadline: 1676145308888, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06. is not online on jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:09,022 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-11 19:54:09,034 INFO [Listener at localhost/38771] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 19:54:09,037 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-11 19:54:09,068 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:09,079 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,084 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,095 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145249095"}]},"ts":"1676145249095"} 2023-02-11 19:54:09,097 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-11 19:54:09,099 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-11 19:54:09,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 19:54:09,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=740882a55424963b0009dd2ab104ea35, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=53477071b15ad5d16d882e1728ca27a7, UNASSIGN}] 2023-02-11 19:54:09,102 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=53477071b15ad5d16d882e1728ca27a7, UNASSIGN 2023-02-11 19:54:09,103 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=53477071b15ad5d16d882e1728ca27a7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:09,104 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145249103"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145249103"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145249103"}]},"ts":"1676145249103"} 2023-02-11 19:54:09,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 53477071b15ad5d16d882e1728ca27a7, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:09,109 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=740882a55424963b0009dd2ab104ea35, UNASSIGN 2023-02-11 19:54:09,110 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=740882a55424963b0009dd2ab104ea35, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:09,110 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145249110"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145249110"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145249110"}]},"ts":"1676145249110"} 2023-02-11 19:54:09,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 740882a55424963b0009dd2ab104ea35, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:09,131 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 19:54:09,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 19:54:09,215 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-11 19:54:09,217 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-11 19:54:09,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:09,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 53477071b15ad5d16d882e1728ca27a7, disabling compactions & flushes 2023-02-11 19:54:09,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:09,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:09,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. after waiting 0 ms 2023-02-11 19:54:09,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:09,270 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-0d256eba79fe4a709594cfe8007c659a, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-077f7aa808be4a158f724a5924d94125, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-e35e7083e26647d8b726dd1e6249a362] to archive 2023-02-11 19:54:09,272 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:09,278 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,280 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,283 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,284 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,287 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-0d256eba79fe4a709594cfe8007c659a to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-0d256eba79fe4a709594cfe8007c659a 2023-02-11 19:54:09,289 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-077f7aa808be4a158f724a5924d94125 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-077f7aa808be4a158f724a5924d94125 2023-02-11 19:54:09,290 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-e35e7083e26647d8b726dd1e6249a362 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-e35e7083e26647d8b726dd1e6249a362 2023-02-11 19:54:09,310 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-2adc5041e32847cd972066f0501b36cc, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-f984e4e0508b431099798dce68b16e5f, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-7a4c27fe15974ec38a0ac636b491797b] to archive 2023-02-11 19:54:09,311 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:09,313 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,314 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,316 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,317 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,319 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-2adc5041e32847cd972066f0501b36cc to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-2adc5041e32847cd972066f0501b36cc 2023-02-11 19:54:09,320 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-f984e4e0508b431099798dce68b16e5f to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-f984e4e0508b431099798dce68b16e5f 2023-02-11 19:54:09,322 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-7a4c27fe15974ec38a0ac636b491797b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/testSplitTableRegionUnevenDaughter=23e1cc59de7125f5225460dcdfdb2b06-7a4c27fe15974ec38a0ac636b491797b 2023-02-11 19:54:09,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:54:09,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:09,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7. 2023-02-11 19:54:09,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 53477071b15ad5d16d882e1728ca27a7: Waiting for close lock at 1676145249258Running coprocessor pre-close hooks at 1676145249258Disabling compacts and flushes for region at 1676145249258Disabling writes for close at 1676145249258Writing region close event to WAL at 1676145249323 (+65 ms)Running coprocessor post-close hooks at 1676145249330 (+7 ms)Closed at 1676145249330 2023-02-11 19:54:09,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:09,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:09,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 740882a55424963b0009dd2ab104ea35, disabling compactions & flushes 2023-02-11 19:54:09,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:09,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:09,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. after waiting 0 ms 2023-02-11 19:54:09,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:09,334 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=53477071b15ad5d16d882e1728ca27a7, regionState=CLOSED 2023-02-11 19:54:09,334 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145249334"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145249334"}]},"ts":"1676145249334"} 2023-02-11 19:54:09,334 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1-bottom] to archive 2023-02-11 19:54:09,335 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:09,339 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/6f2473a6ea6b46138b82b2974897289b.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,342 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/de47f19d894d43bbb2e5b54456cdfcaf.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-11 19:54:09,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 53477071b15ad5d16d882e1728ca27a7, server=jenkins-hbase4.apache.org,42733,1676145241889 in 232 msec 2023-02-11 19:54:09,344 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/1e7751d340aa4cd7abb5a88e1f769996.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=53477071b15ad5d16d882e1728ca27a7, UNASSIGN in 242 msec 2023-02-11 19:54:09,347 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/57c47c5fdf4448a3bd13a506d60130a1.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,349 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082-bottom] to archive 2023-02-11 19:54:09,351 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:09,355 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/7a99efe5ca7b4d5da2db8038ad247ddf.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,357 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/b808ed1892d04ad49703d51807c592f8.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,359 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/e2dd487614e343469ce250d0764fb4ea.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,362 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/676e8ab1791c432faacef36fb7c06082.23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:54:09,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:09,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35. 2023-02-11 19:54:09,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 740882a55424963b0009dd2ab104ea35: Waiting for close lock at 1676145249333Running coprocessor pre-close hooks at 1676145249333Disabling compacts and flushes for region at 1676145249333Disabling writes for close at 1676145249333Writing region close event to WAL at 1676145249363 (+30 ms)Running coprocessor post-close hooks at 1676145249371 (+8 ms)Closed at 1676145249373 (+2 ms) 2023-02-11 19:54:09,375 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:09,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=740882a55424963b0009dd2ab104ea35, regionState=CLOSED 2023-02-11 19:54:09,376 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676145249376"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145249376"}]},"ts":"1676145249376"} 2023-02-11 19:54:09,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-11 19:54:09,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 740882a55424963b0009dd2ab104ea35, server=jenkins-hbase4.apache.org,42733,1676145241889 in 266 msec 2023-02-11 19:54:09,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-11 19:54:09,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=740882a55424963b0009dd2ab104ea35, UNASSIGN in 281 msec 2023-02-11 19:54:09,385 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145249385"}]},"ts":"1676145249385"} 2023-02-11 19:54:09,387 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-11 19:54:09,389 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-11 19:54:09,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 304 msec 2023-02-11 19:54:09,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 19:54:09,407 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-11 19:54:09,414 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,425 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,427 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 19:54:09,436 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,436 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:09,436 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:09,441 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/recovered.edits] 2023-02-11 19:54:09,441 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/recovered.edits] 2023-02-11 19:54:09,441 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/recovered.edits] 2023-02-11 19:54:09,446 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/ffe33da3dd8e43e2ab09d58982d69004 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf1/ffe33da3dd8e43e2ab09d58982d69004 2023-02-11 19:54:09,447 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/077f7aa808be4a158f724a5924d94125 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/077f7aa808be4a158f724a5924d94125 2023-02-11 19:54:09,447 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/7e8ab368bb8e4b7bbf33deadf34efe65 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf1/7e8ab368bb8e4b7bbf33deadf34efe65 2023-02-11 19:54:09,449 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/0d256eba79fe4a709594cfe8007c659a to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/0d256eba79fe4a709594cfe8007c659a 2023-02-11 19:54:09,450 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/dce7bebacae94b82b39484403edac6ab to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/cf2/dce7bebacae94b82b39484403edac6ab 2023-02-11 19:54:09,451 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/1e7751d340aa4cd7abb5a88e1f769996 2023-02-11 19:54:09,451 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/dc37cd42d3f3446da0d7aac190381df3 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/cf2/dc37cd42d3f3446da0d7aac190381df3 2023-02-11 19:54:09,453 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/57c47c5fdf4448a3bd13a506d60130a1 2023-02-11 19:54:09,455 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7/recovered.edits/90.seqid 2023-02-11 19:54:09,455 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/6f2473a6ea6b46138b82b2974897289b 2023-02-11 19:54:09,456 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35/recovered.edits/90.seqid 2023-02-11 19:54:09,457 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:09,458 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/740882a55424963b0009dd2ab104ea35 2023-02-11 19:54:09,459 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/de47f19d894d43bbb2e5b54456cdfcaf 2023-02-11 19:54:09,461 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/e35e7083e26647d8b726dd1e6249a362 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf1/e35e7083e26647d8b726dd1e6249a362 2023-02-11 19:54:09,465 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/2adc5041e32847cd972066f0501b36cc to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/2adc5041e32847cd972066f0501b36cc 2023-02-11 19:54:09,468 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/676e8ab1791c432faacef36fb7c06082 2023-02-11 19:54:09,470 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a4c27fe15974ec38a0ac636b491797b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a4c27fe15974ec38a0ac636b491797b 2023-02-11 19:54:09,472 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/7a99efe5ca7b4d5da2db8038ad247ddf 2023-02-11 19:54:09,474 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/b808ed1892d04ad49703d51807c592f8 2023-02-11 19:54:09,475 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/e2dd487614e343469ce250d0764fb4ea 2023-02-11 19:54:09,477 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/f984e4e0508b431099798dce68b16e5f to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/cf2/f984e4e0508b431099798dce68b16e5f 2023-02-11 19:54:09,483 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/recovered.edits/85.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06/recovered.edits/85.seqid 2023-02-11 19:54:09,484 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionUnevenDaughter/23e1cc59de7125f5225460dcdfdb2b06 2023-02-11 19:54:09,484 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-11 19:54:09,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,511 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-11 19:54:09,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 19:54:09,558 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-11 19:54:09,560 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,560 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-11 19:54:09,560 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145249560"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:09,561 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145249560"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:09,561 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145249560"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:09,564 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 19:54:09,565 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 23e1cc59de7125f5225460dcdfdb2b06, NAME => 'testSplitTableRegionUnevenDaughter,,1676145244722.23e1cc59de7125f5225460dcdfdb2b06.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 740882a55424963b0009dd2ab104ea35, NAME => 'testSplitTableRegionUnevenDaughter,,1676145246874.740882a55424963b0009dd2ab104ea35.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 53477071b15ad5d16d882e1728ca27a7, NAME => 'testSplitTableRegionUnevenDaughter,26,1676145246874.53477071b15ad5d16d882e1728ca27a7.', STARTKEY => '26', ENDKEY => ''}] 2023-02-11 19:54:09,565 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-11 19:54:09,565 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145249565"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:09,567 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-11 19:54:09,569 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:09,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 154 msec 2023-02-11 19:54:09,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 19:54:09,733 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-11 19:54:09,752 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=390 (was 358) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33854 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:39438 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36642 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:39166 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36632 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33770 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33894 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36636 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36654 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33968 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:34260 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5876c91-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5876c91-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5876c91-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36688 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:42733-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36650 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5876c91-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5876c91-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36718 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:39454 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36364 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=832 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=437 (was 275) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3879 (was 4003) 2023-02-11 19:54:09,766 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=390, OpenFileDescriptor=832, MaxFileDescriptor=60000, SystemLoadAverage=437, ProcessCount=168, AvailableMemoryMB=3878 2023-02-11 19:54:09,767 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:09,772 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:54:09,772 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:54:09,774 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:09,776 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:54:09,777 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:54:09,778 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d 2023-02-11 19:54:09,779 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d empty. 2023-02-11 19:54:09,779 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d 2023-02-11 19:54:09,779 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-11 19:54:09,791 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:09,792 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 281aa262152bf186516a3159abcf093d, NAME => 'testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:54:09,802 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:09,803 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 281aa262152bf186516a3159abcf093d, disabling compactions & flushes 2023-02-11 19:54:09,803 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:09,803 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:09,803 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. after waiting 0 ms 2023-02-11 19:54:09,803 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:09,803 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:09,803 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 281aa262152bf186516a3159abcf093d: Waiting for close lock at 1676145249803Disabling compacts and flushes for region at 1676145249803Disabling writes for close at 1676145249803Writing region close event to WAL at 1676145249803Closed at 1676145249803 2023-02-11 19:54:09,806 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:54:09,807 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145249806"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145249806"}]},"ts":"1676145249806"} 2023-02-11 19:54:09,809 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:54:09,810 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:54:09,810 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145249810"}]},"ts":"1676145249810"} 2023-02-11 19:54:09,812 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-11 19:54:09,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:09,817 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:09,817 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:09,817 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:09,817 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:54:09,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, ASSIGN}] 2023-02-11 19:54:09,819 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, ASSIGN 2023-02-11 19:54:09,820 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:09,970 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:09,971 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=281aa262152bf186516a3159abcf093d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:09,971 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145249971"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145249971"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145249971"}]},"ts":"1676145249971"} 2023-02-11 19:54:09,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:10,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 281aa262152bf186516a3159abcf093d, NAME => 'testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:10,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:10,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:10,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:10,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:10,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:10,132 INFO [StoreOpener-281aa262152bf186516a3159abcf093d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:10,134 DEBUG [StoreOpener-281aa262152bf186516a3159abcf093d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1 2023-02-11 19:54:10,134 DEBUG [StoreOpener-281aa262152bf186516a3159abcf093d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1 2023-02-11 19:54:10,134 INFO [StoreOpener-281aa262152bf186516a3159abcf093d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 281aa262152bf186516a3159abcf093d columnFamilyName cf1 2023-02-11 19:54:10,135 INFO [StoreOpener-281aa262152bf186516a3159abcf093d-1] regionserver.HStore(310): Store=281aa262152bf186516a3159abcf093d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:10,135 INFO [StoreOpener-281aa262152bf186516a3159abcf093d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:10,137 DEBUG [StoreOpener-281aa262152bf186516a3159abcf093d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2 2023-02-11 19:54:10,137 DEBUG [StoreOpener-281aa262152bf186516a3159abcf093d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2 2023-02-11 19:54:10,138 INFO [StoreOpener-281aa262152bf186516a3159abcf093d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 281aa262152bf186516a3159abcf093d columnFamilyName cf2 2023-02-11 19:54:10,138 INFO [StoreOpener-281aa262152bf186516a3159abcf093d-1] regionserver.HStore(310): Store=281aa262152bf186516a3159abcf093d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:10,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d 2023-02-11 19:54:10,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d 2023-02-11 19:54:10,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:10,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:10,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:10,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 281aa262152bf186516a3159abcf093d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74356238, jitterRate=0.10799428820610046}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:10,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 281aa262152bf186516a3159abcf093d: Running coprocessor pre-open hook at 1676145250131Writing region info on filesystem at 1676145250131Initializing all the Stores at 1676145250131Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145250132 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145250132Cleaning up temporary data from old regions at 1676145250140 (+8 ms)Cleaning up detritus from prior splits at 1676145250141 (+1 ms)Running coprocessor post-open hooks at 1676145250145 (+4 ms)Region opened successfully at 1676145250145 2023-02-11 19:54:10,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d., pid=27, masterSystemTime=1676145250126 2023-02-11 19:54:10,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,149 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=281aa262152bf186516a3159abcf093d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,149 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145250149"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145250149"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145250149"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145250149"}]},"ts":"1676145250149"} 2023-02-11 19:54:10,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-11 19:54:10,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 in 177 msec 2023-02-11 19:54:10,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-11 19:54:10,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, ASSIGN in 337 msec 2023-02-11 19:54:10,157 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:54:10,157 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145250157"}]},"ts":"1676145250157"} 2023-02-11 19:54:10,159 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-11 19:54:10,161 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:54:10,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 388 msec 2023-02-11 19:54:10,285 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:10,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 19:54:10,287 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 19:54:10,287 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,288 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 19:54:10,288 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 19:54:10,288 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,289 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,292 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,292 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,292 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,292 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,293 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,293 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:10,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,294 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,294 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,295 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,295 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,296 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,296 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,296 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,296 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,296 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,297 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:10,297 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,297 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,298 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. started... 2023-02-11 19:54:10,297 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:10,298 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,298 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,298 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 281aa262152bf186516a3159abcf093d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 19:54:10,299 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,301 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,301 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,301 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,301 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,301 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,312 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/c20b827a189a4cc7992ef377dd1243eb 2023-02-11 19:54:10,329 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/b41291a2e3d1472b9fd125a0732778eb 2023-02-11 19:54:10,334 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/c20b827a189a4cc7992ef377dd1243eb as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb 2023-02-11 19:54:10,340 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:54:10,341 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/b41291a2e3d1472b9fd125a0732778eb as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb 2023-02-11 19:54:10,346 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:54:10,347 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 281aa262152bf186516a3159abcf093d in 49ms, sequenceid=6, compaction requested=false 2023-02-11 19:54:10,347 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-11 19:54:10,348 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 281aa262152bf186516a3159abcf093d: 2023-02-11 19:54:10,348 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,350 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,350 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,350 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,350 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 19:54:10,352 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 19:54:10,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@546eb30d[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:10,352 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,357 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,358 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,358 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,359 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,359 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:10,363 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 19:54:10,363 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 19:54:10,363 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:10,363 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:10,363 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 19:54:10,363 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:10,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,366 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,366 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,366 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,367 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,369 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,369 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,369 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,375 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,375 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,375 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,375 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,375 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,375 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,375 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,375 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,376 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,376 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,375 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 19:54:10,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,375 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,376 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:10,376 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,377 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,377 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,377 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:10,377 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,377 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:10,377 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,378 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,378 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,478 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:10,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:10,497 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:10,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:10,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 19:54:10,499 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 19:54:10,499 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 19:54:10,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 19:54:10,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,502 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,502 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,502 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,502 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,502 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,502 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:10,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,503 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,504 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,504 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,504 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,504 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,504 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,504 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,504 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,504 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,504 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,504 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,504 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,504 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,505 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,505 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,505 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,505 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,505 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,505 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,505 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,505 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,505 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:10,506 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:10,506 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. started... 2023-02-11 19:54:10,506 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 281aa262152bf186516a3159abcf093d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:10,507 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,507 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,507 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,507 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,507 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,507 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,519 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/47215e09f5dd47d8ae1d858013fcba7d 2023-02-11 19:54:10,536 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/556081219aea44fab0e30a14389b8a24 2023-02-11 19:54:10,543 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/47215e09f5dd47d8ae1d858013fcba7d as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d 2023-02-11 19:54:10,548 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:54:10,552 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/556081219aea44fab0e30a14389b8a24 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24 2023-02-11 19:54:10,558 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:54:10,559 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 281aa262152bf186516a3159abcf093d in 53ms, sequenceid=19, compaction requested=false 2023-02-11 19:54:10,559 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 281aa262152bf186516a3159abcf093d: 2023-02-11 19:54:10,559 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,562 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,562 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,562 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,562 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 19:54:10,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@316d03d8[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:10,564 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 19:54:10,565 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,566 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,566 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,566 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:10,570 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,570 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,570 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,570 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,571 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,571 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,572 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,572 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,572 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,573 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:10,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 19:54:10,576 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 19:54:10,576 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:10,576 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:10,576 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 19:54:10,576 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:10,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,578 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,579 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,580 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,583 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,583 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,584 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,588 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 19:54:10,589 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:10,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,588 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,589 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:10,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,589 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:10,689 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:10,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:10,710 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:10,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:10,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 19:54:10,711 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 19:54:10,712 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,712 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 19:54:10,712 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 19:54:10,713 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,713 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,716 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,716 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,716 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,716 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,716 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,716 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:10,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,718 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,718 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,718 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,718 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,718 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,719 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,719 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,719 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,718 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,720 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,720 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,720 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,720 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:10,720 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:10,720 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. started... 2023-02-11 19:54:10,720 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,720 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,720 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 281aa262152bf186516a3159abcf093d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:10,721 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,722 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,722 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,722 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,723 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,723 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,731 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/6fe312ca09454c359047d18c10ebeb14 2023-02-11 19:54:10,750 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 2023-02-11 19:54:10,755 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/6fe312ca09454c359047d18c10ebeb14 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14 2023-02-11 19:54:10,760 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:54:10,761 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 2023-02-11 19:54:10,766 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:54:10,767 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 281aa262152bf186516a3159abcf093d in 47ms, sequenceid=32, compaction requested=true 2023-02-11 19:54:10,767 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 281aa262152bf186516a3159abcf093d: 2023-02-11 19:54:10,768 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,768 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:10,768 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:10,768 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,768 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,768 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,769 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,769 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,770 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,770 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 19:54:10,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77a0ec58[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:10,772 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 19:54:10,772 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,774 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,774 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,774 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,775 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,775 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,775 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:10,777 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,777 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,777 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,778 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,778 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,778 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,778 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,778 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,779 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,779 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:10,783 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 19:54:10,783 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 19:54:10,783 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:10,783 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:10,783 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 19:54:10,783 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:10,785 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,785 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,785 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,786 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,787 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,787 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,789 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,790 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,790 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,795 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,795 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 19:54:10,796 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,796 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,796 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:10,796 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,795 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,796 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,796 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:10,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,797 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,797 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,797 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,821 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 19:54:10,822 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-11 19:54:10,822 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:10,823 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-11 19:54:10,897 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:10,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:10,916 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:10,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:10,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 19:54:10,920 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 19:54:10,920 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,921 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 19:54:10,921 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 19:54:10,921 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,921 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,923 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,923 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,923 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,923 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,923 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,923 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,924 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,924 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:10,926 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:10,925 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,926 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:10,926 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:10,926 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,926 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:10,926 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:10,926 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,928 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:10,928 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. started... 2023-02-11 19:54:10,926 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,928 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,928 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,928 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 281aa262152bf186516a3159abcf093d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:10,932 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,939 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,939 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,939 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,939 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,939 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,939 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/bc946d87bea9492091d0bf464b3f37ec 2023-02-11 19:54:10,954 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/588e8ee6e523402c8ca95fe75beb8cba 2023-02-11 19:54:10,959 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/bc946d87bea9492091d0bf464b3f37ec as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec 2023-02-11 19:54:10,964 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:54:10,964 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/588e8ee6e523402c8ca95fe75beb8cba as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba 2023-02-11 19:54:10,969 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:54:10,970 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 281aa262152bf186516a3159abcf093d in 42ms, sequenceid=45, compaction requested=true 2023-02-11 19:54:10,970 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 281aa262152bf186516a3159abcf093d: 2023-02-11 19:54:10,971 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:10,971 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:10,971 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:10,971 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,971 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:10,971 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:10,972 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,972 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,973 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,973 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:10,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 19:54:10,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d9a03aa[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:10,975 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 19:54:10,975 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,976 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:10,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,976 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,976 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:10,976 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,976 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:10,977 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:10,976 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:10,977 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:10,978 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,978 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,978 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,978 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,979 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,978 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,980 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:10,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,979 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,980 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,980 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:10,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:10,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 19:54:10,985 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 19:54:10,985 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:10,985 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:10,985 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 19:54:10,985 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:10,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,988 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:10,989 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:10,990 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,990 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:10,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,991 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:10,992 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:10,992 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:10,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,995 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 19:54:10,996 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:10,995 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:10,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:10,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:10,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:10,996 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:10,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:10,996 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:10,996 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:10,996 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,996 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:10,996 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:10,997 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:10,997 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:10,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:10,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,096 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:11,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:11,116 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:11,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:11,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 19:54:11,118 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 19:54:11,118 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:11,118 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 19:54:11,118 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 19:54:11,119 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,119 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,120 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,120 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,120 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,120 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,121 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,121 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:11,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,121 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,122 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:11,122 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:11,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:11,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:11,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:11,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:11,123 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:11,123 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. started... 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:11,123 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:11,124 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 281aa262152bf186516a3159abcf093d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:11,125 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,125 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,125 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,126 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:11,126 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,126 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:11,137 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/42a205d1c88b498d9c9f013593e524d7 2023-02-11 19:54:11,151 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/0618e56885e7491ebfe5f05cc094fb3b 2023-02-11 19:54:11,157 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/42a205d1c88b498d9c9f013593e524d7 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7 2023-02-11 19:54:11,162 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:54:11,163 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/0618e56885e7491ebfe5f05cc094fb3b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b 2023-02-11 19:54:11,167 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:54:11,169 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 281aa262152bf186516a3159abcf093d in 45ms, sequenceid=58, compaction requested=true 2023-02-11 19:54:11,169 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 281aa262152bf186516a3159abcf093d: 2023-02-11 19:54:11,169 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:11,169 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:11,169 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:11,169 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,169 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:11,169 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:11,171 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,171 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:11,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:11,172 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,172 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:11,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:11,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:11,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:11,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 19:54:11,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ccd37cf[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:11,174 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 19:54:11,174 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:11,175 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:11,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,176 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:11,176 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:11,176 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:11,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,175 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,176 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:11,176 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:11,176 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:11,176 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:11,176 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:11,177 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:11,177 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:11,177 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:11,178 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:11,178 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:11,178 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:11,185 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:11,185 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:11,185 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,185 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:11,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:11,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:11,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:11,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:11,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:11,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:11,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 19:54:11,188 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 19:54:11,188 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:11,188 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:11,188 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 19:54:11,188 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:11,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:11,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:11,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:11,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,190 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,191 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,191 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:11,193 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,193 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,193 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,199 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:11,199 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,199 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 19:54:11,199 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:11,199 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,199 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,199 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,199 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,199 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:11,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,199 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,199 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:11,200 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,200 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,300 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:11,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:11,319 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:11,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:11,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 19:54:11,321 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 19:54:11,321 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:11,321 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 19:54:11,321 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 19:54:11,322 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,322 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,324 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,324 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,324 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,324 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,324 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,324 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:11,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,325 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:11,326 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:11,326 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:11,326 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:11,326 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:11,326 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:11,327 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 19:54:11,327 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:11,327 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:11,327 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:11,327 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:11,327 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:11,327 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. started... 2023-02-11 19:54:11,327 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 281aa262152bf186516a3159abcf093d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:11,328 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,328 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,328 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:11,329 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,329 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,329 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:11,339 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/97846742557342ed85c83d1040a33c5b 2023-02-11 19:54:11,359 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/1143961ec97c46f19aa196e0bf65e2a1 2023-02-11 19:54:11,364 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/97846742557342ed85c83d1040a33c5b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b 2023-02-11 19:54:11,369 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:54:11,370 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/1143961ec97c46f19aa196e0bf65e2a1 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1 2023-02-11 19:54:11,375 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:54:11,376 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 281aa262152bf186516a3159abcf093d in 49ms, sequenceid=71, compaction requested=true 2023-02-11 19:54:11,376 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 281aa262152bf186516a3159abcf093d: 2023-02-11 19:54:11,376 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:11,376 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:11,376 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:11,376 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,376 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 19:54:11,376 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 19:54:11,378 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,378 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:11,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:11,378 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,378 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:11,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:11,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:11,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:11,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 19:54:11,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35678a4d[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:11,380 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 19:54:11,380 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,383 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:11,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,383 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:11,384 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:11,384 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:11,384 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 19:54:11,384 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:11,384 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:11,384 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 19:54:11,384 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:11,384 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:11,385 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:11,385 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:11,385 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:11,386 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:11,386 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,386 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 19:54:11,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:11,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:11,386 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:11,386 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:11,386 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:11,386 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 19:54:11,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:11,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:11,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:11,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:11,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 19:54:11,389 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 19:54:11,389 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:11,389 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:11,389 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 19:54:11,389 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:11,391 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,391 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:11,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:11,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,391 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,391 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:11,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,392 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,392 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:11,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,394 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:11,394 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 19:54:11,395 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,398 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:11,398 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 19:54:11,398 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:11,399 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:11,399 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:11,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 19:54:11,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:11,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:11,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:11,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:11,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:11,400 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,400 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:11,400 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 19:54:11,400 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 19:54:11,499 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:11,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:11,765 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 19:54:11,766 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:11,767 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:11,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:11,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:11,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:11,770 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:11,770 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:11,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:11,770 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:12,018 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,019 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:12,019 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:12,019 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:12,021 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-11 19:54:12,022 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:12,022 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:12,022 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:12,022 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:12,023 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:12,027 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 19:54:12,029 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:12,029 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:12,029 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:12,030 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:12,030 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,031 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:12,031 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,031 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,032 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:12,032 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,033 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:12,033 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,033 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,033 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,033 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-11 19:54:12,034 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:12,034 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:12,035 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145282035 2023-02-11 19:54:12,037 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:12,037 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,041 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:12,041 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:12,041 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:12,041 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145312041 2023-02-11 19:54:12,042 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145372042 2023-02-11 19:54:12,042 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:54:12,042 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:12,044 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:12,045 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:12,045 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:12,292 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,292 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:12,292 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:12,292 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:12,293 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:12,293 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:12,293 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:12,294 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:12,294 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:12,294 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:12,296 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 19:54:12,297 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:12,298 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:12,298 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:12,298 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:12,298 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,299 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:12,299 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,299 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,299 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:12,300 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,300 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:12,300 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,300 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,300 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,300 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:12,301 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:12,301 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:12,303 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145282303 2023-02-11 19:54:12,304 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,305 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:12,307 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:12,307 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:12,308 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:12,308 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145312308 2023-02-11 19:54:12,308 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145372308 2023-02-11 19:54:12,308 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:12,308 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:12,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN}] 2023-02-11 19:54:12,310 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:12,310 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:12,310 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:12,558 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,558 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:12,558 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:12,558 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:12,559 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:12,560 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:12,560 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:12,560 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:12,560 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:12,561 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:12,562 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 19:54:12,562 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:12,563 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:12,563 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:12,563 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:12,564 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,564 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:12,565 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,565 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,565 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:12,565 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,566 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:12,566 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,566 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,566 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,566 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:12,567 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:12,567 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:12,570 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145282570 2023-02-11 19:54:12,571 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,571 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:12,574 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:12,574 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:12,574 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:12,574 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145312574 2023-02-11 19:54:12,574 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145372574 2023-02-11 19:54:12,574 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:12,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN}] 2023-02-11 19:54:12,575 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:12,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:12,578 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=281aa262152bf186516a3159abcf093d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:12,578 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145252578"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145252578"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145252578"}]},"ts":"1676145252578"} 2023-02-11 19:54:12,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:12,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:12,580 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:12,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:12,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:12,825 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,825 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:12,825 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:12,825 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:12,825 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:12,825 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:12,825 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:12,825 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:12,825 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:12,826 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:12,828 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:12,828 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 19:54:12,828 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:12,828 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:12,829 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:12,829 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:12,830 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,830 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:12,830 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,830 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:12,831 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:12,832 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,832 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:12,832 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:12,832 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,832 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,832 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:12,832 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:12,832 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:12,832 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:12,833 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:12,833 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:12,834 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d to restore RIT 2023-02-11 19:54:12,834 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145282834 2023-02-11 19:54:12,835 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:12,835 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:12,837 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:12,838 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:12,838 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:12,838 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145312838 2023-02-11 19:54:12,838 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145372838 2023-02-11 19:54:12,838 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:12,838 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=281aa262152bf186516a3159abcf093d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:12,839 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145252838"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145252838"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145252838"}]},"ts":"1676145252838"} 2023-02-11 19:54:12,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:12,840 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:12,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:12,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 281aa262152bf186516a3159abcf093d, disabling compactions & flushes 2023-02-11 19:54:12,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:12,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:12,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. after waiting 0 ms 2023-02-11 19:54:12,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:12,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 281aa262152bf186516a3159abcf093d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 19:54:13,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/9123fe3aeef24307b6a1ca20ccf3dbba 2023-02-11 19:54:13,022 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/0e876483ccc246ca8479eb1cd3e03d18 2023-02-11 19:54:13,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf1/9123fe3aeef24307b6a1ca20ccf3dbba as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba 2023-02-11 19:54:13,032 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:54:13,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/.tmp/cf2/0e876483ccc246ca8479eb1cd3e03d18 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18 2023-02-11 19:54:13,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:54:13,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 281aa262152bf186516a3159abcf093d in 44ms, sequenceid=82, compaction requested=true 2023-02-11 19:54:13,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 19:54:13,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:13,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. 2023-02-11 19:54:13,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 281aa262152bf186516a3159abcf093d: Waiting for close lock at 1676145252993Running coprocessor pre-close hooks at 1676145252993Disabling compacts and flushes for region at 1676145252993Disabling writes for close at 1676145252994 (+1 ms)Obtaining lock to block concurrent updates at 1676145252994Preparing flush snapshotting stores in 281aa262152bf186516a3159abcf093d at 1676145252994Finished memstore snapshotting testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676145252994Flushing stores of testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. at 1676145252994Flushing 281aa262152bf186516a3159abcf093d/cf1: creating writer at 1676145252995 (+1 ms)Flushing 281aa262152bf186516a3159abcf093d/cf1: appending metadata at 1676145252998 (+3 ms)Flushing 281aa262152bf186516a3159abcf093d/cf1: closing flushed file at 1676145252998Flushing 281aa262152bf186516a3159abcf093d/cf2: creating writer at 1676145253011 (+13 ms)Flushing 281aa262152bf186516a3159abcf093d/cf2: appending metadata at 1676145253014 (+3 ms)Flushing 281aa262152bf186516a3159abcf093d/cf2: closing flushed file at 1676145253014Flushing 281aa262152bf186516a3159abcf093d/cf1: reopening flushed file at 1676145253028 (+14 ms)Flushing 281aa262152bf186516a3159abcf093d/cf2: reopening flushed file at 1676145253033 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 281aa262152bf186516a3159abcf093d in 44ms, sequenceid=82, compaction requested=true at 1676145253038 (+5 ms)Writing region close event to WAL at 1676145253054 (+16 ms)Running coprocessor post-close hooks at 1676145253058 (+4 ms)Closed at 1676145253058 2023-02-11 19:54:13,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,061 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=281aa262152bf186516a3159abcf093d, regionState=CLOSED 2023-02-11 19:54:13,061 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145253061"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145253061"}]},"ts":"1676145253061"} 2023-02-11 19:54:13,063 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:13,063 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:13,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:13,063 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:13,088 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,088 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:13,088 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:13,088 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:13,089 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:13,089 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:13,089 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:13,089 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:13,089 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:13,090 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:13,091 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:13,091 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 19:54:13,091 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-11 19:54:13,091 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:13,092 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:13,092 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:13,092 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:13,092 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,093 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:13,093 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,093 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,094 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:13,094 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,094 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:13,094 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:13,095 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:13,095 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,095 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,095 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,095 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:13,095 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:13,095 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:13,095 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:13,096 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:13,096 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:13,096 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d to restore RIT 2023-02-11 19:54:13,097 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145283097 2023-02-11 19:54:13,097 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:13,097 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:13,100 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:13,100 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:13,100 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:13,100 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145313100 2023-02-11 19:54:13,101 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145373101 2023-02-11 19:54:13,101 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:13,101 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=281aa262152bf186516a3159abcf093d, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:13,101 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145253101"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145253101"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145253101"}]},"ts":"1676145253101"} 2023-02-11 19:54:13,103 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:13,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-11 19:54:13,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 in 263 msec 2023-02-11 19:54:13,106 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:13,106 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:13,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:13,107 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:13,351 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,351 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:13,351 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:13,351 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:13,352 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:13,353 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:13,353 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:13,353 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:13,353 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:13,353 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:13,355 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:13,355 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 19:54:13,355 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:13,355 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:13,356 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:13,356 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:13,356 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:13,357 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,357 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:13,357 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,357 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,358 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:13,359 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,359 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:13,359 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:13,359 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:13,359 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,359 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,359 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,359 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:13,359 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:13,359 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:13,360 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:13,360 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:13,361 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d to restore RIT 2023-02-11 19:54:13,361 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145283361 2023-02-11 19:54:13,362 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:13,362 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:13,364 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:13,364 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:13,365 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:13,365 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145313365 2023-02-11 19:54:13,365 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145373365 2023-02-11 19:54:13,365 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:54:13,365 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:13,366 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:13,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-11 19:54:13,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN in 791 msec 2023-02-11 19:54:13,372 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:13,372 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:13,372 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:13,615 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,615 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:13,615 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:13,615 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:13,615 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:13,616 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:13,616 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:13,616 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:13,616 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:13,617 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:13,618 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:13,618 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:13,618 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:13,618 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:13,619 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:13,619 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:13,619 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:13,619 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,620 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:13,620 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,620 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:13,621 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:13,621 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,621 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:13,621 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:13,621 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:13,621 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:13,622 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:13,622 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:13,622 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:13,623 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:13,623 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145283623 2023-02-11 19:54:13,624 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:13,624 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,627 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:13,627 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:13,627 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:13,627 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145313627 2023-02-11 19:54:13,627 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145373627 2023-02-11 19:54:13,627 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:54:13,627 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,628 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,628 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,632 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:13,642 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=281aa262152bf186516a3159abcf093d, threads=14 2023-02-11 19:54:13,643 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,643 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,643 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,643 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,643 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,643 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,643 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,644 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,644 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,644 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,644 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,650 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,650 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,650 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,795 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,805 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,812 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,822 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,823 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,826 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,826 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,826 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,827 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,830 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,835 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,842 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,845 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,846 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:13,846 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 281aa262152bf186516a3159abcf093d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:54:13,880 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:13,880 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:13,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:13,880 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:13,970 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 19:54:14,128 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,128 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:14,128 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:14,128 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:14,128 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:14,128 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:14,128 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:14,128 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:14,128 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:14,129 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:14,130 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 19:54:14,130 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:14,130 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:14,131 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:14,131 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:14,131 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:14,132 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:14,132 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:14,133 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:14,133 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:14,133 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:14,134 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:14,134 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,134 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:14,134 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:14,135 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:14,135 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,135 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,135 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,135 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:14,135 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:14,135 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:14,136 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:14,136 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:14,136 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145284136 2023-02-11 19:54:14,137 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:14,138 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,140 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:14,140 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:14,140 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:14,141 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145314141 2023-02-11 19:54:14,141 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145374141 2023-02-11 19:54:14,141 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:54:14,150 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=281aa262152bf186516a3159abcf093d, threads=14 2023-02-11 19:54:14,151 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,151 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,151 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,152 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,152 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,152 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,152 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,155 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,156 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,156 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,156 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,157 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,160 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,160 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,275 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,287 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,313 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,313 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,335 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,339 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,339 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,374 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,377 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,384 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,384 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,385 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,789 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,790 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba for region: 281aa262152bf186516a3159abcf093d 2023-02-11 19:54:14,791 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 281aa262152bf186516a3159abcf093d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:54:14,829 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:14,834 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:54:14,836 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:54:14,836 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:14,836 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:14,836 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:14,892 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,892 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:14,893 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:14,893 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:14,893 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:14,893 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:14,893 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:14,893 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:14,893 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:14,894 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:14,895 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 19:54:14,895 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:14,895 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:14,895 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:14,896 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:14,896 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:14,896 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:14,897 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:14,897 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:14,897 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:14,897 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:14,898 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:14,898 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,898 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:14,898 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:14,898 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:14,899 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,899 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,899 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,899 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:14,899 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:14,899 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:14,900 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:14,900 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:14,901 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145284901 2023-02-11 19:54:14,902 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,902 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:14,904 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:14,905 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:14,905 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:14,905 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145314905 2023-02-11 19:54:14,905 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145374905 2023-02-11 19:54:14,905 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:14,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:14,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:14,910 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:14,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:14,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:15,155 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:15,156 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:15,156 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:15,156 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:15,156 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:15,156 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:15,156 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:15,156 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:15,156 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:15,157 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:15,158 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 19:54:15,158 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:15,158 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:15,158 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:15,159 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:15,159 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:15,159 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:15,159 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:15,160 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:15,160 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:15,160 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:15,160 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:15,161 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,161 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN 2023-02-11 19:54:15,161 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:15,161 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:15,161 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,161 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,161 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,161 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=281aa262152bf186516a3159abcf093d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:15,161 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 281aa262152bf186516a3159abcf093d, server=jenkins-hbase4.apache.org,42733,1676145241889 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:15,162 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:15,162 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:15,163 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:15,163 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145285163 2023-02-11 19:54:15,163 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:15,165 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,167 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:15,167 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=281aa262152bf186516a3159abcf093d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:15,167 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:15,167 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145315167 2023-02-11 19:54:15,167 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145375167 2023-02-11 19:54:15,167 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:54:15,169 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145255169"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676145255169"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676145255169"}]},"ts":"1676145255169"} 2023-02-11 19:54:15,169 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145255169"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145255169"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145255169"}]},"ts":"1676145255169"} 2023-02-11 19:54:15,170 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145255169"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145255169"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145255169"}]},"ts":"1676145255169"} 2023-02-11 19:54:15,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0f38e9ae28b3a3299ac12ff8d13dfd48, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fdf3f42835ea1d9ba8f222165d44edd, ASSIGN}] 2023-02-11 19:54:15,176 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fdf3f42835ea1d9ba8f222165d44edd, ASSIGN 2023-02-11 19:54:15,177 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fdf3f42835ea1d9ba8f222165d44edd, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:15,177 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0f38e9ae28b3a3299ac12ff8d13dfd48, ASSIGN 2023-02-11 19:54:15,178 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0f38e9ae28b3a3299ac12ff8d13dfd48, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:15,327 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 19:54:15,328 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=0f38e9ae28b3a3299ac12ff8d13dfd48, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:15,329 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145255328"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145255328"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145255328"}]},"ts":"1676145255328"} 2023-02-11 19:54:15,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 0f38e9ae28b3a3299ac12ff8d13dfd48, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:15,334 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=4fdf3f42835ea1d9ba8f222165d44edd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:15,334 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145255334"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145255334"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145255334"}]},"ts":"1676145255334"} 2023-02-11 19:54:15,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 4fdf3f42835ea1d9ba8f222165d44edd, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:15,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:15,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0f38e9ae28b3a3299ac12ff8d13dfd48, NAME => 'testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 19:54:15,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:15,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:15,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:15,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:15,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:15,488 INFO [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:15,489 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1 2023-02-11 19:54:15,490 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1 2023-02-11 19:54:15,490 INFO [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f38e9ae28b3a3299ac12ff8d13dfd48 columnFamilyName cf1 2023-02-11 19:54:15,506 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7-bottom 2023-02-11 19:54:15,511 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d-bottom 2023-02-11 19:54:15,516 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14-bottom 2023-02-11 19:54:15,522 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba-bottom 2023-02-11 19:54:15,527 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b-bottom 2023-02-11 19:54:15,532 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec-bottom 2023-02-11 19:54:15,536 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb-bottom 2023-02-11 19:54:15,536 INFO [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(310): Store=0f38e9ae28b3a3299ac12ff8d13dfd48/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:15,536 INFO [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:15,537 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2 2023-02-11 19:54:15,537 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2 2023-02-11 19:54:15,538 INFO [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f38e9ae28b3a3299ac12ff8d13dfd48 columnFamilyName cf2 2023-02-11 19:54:15,552 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b-bottom 2023-02-11 19:54:15,557 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18-bottom 2023-02-11 19:54:15,561 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1-bottom 2023-02-11 19:54:15,566 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24-bottom 2023-02-11 19:54:15,570 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba-bottom 2023-02-11 19:54:15,576 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1-bottom 2023-02-11 19:54:15,580 DEBUG [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb-bottom 2023-02-11 19:54:15,581 INFO [StoreOpener-0f38e9ae28b3a3299ac12ff8d13dfd48-1] regionserver.HStore(310): Store=0f38e9ae28b3a3299ac12ff8d13dfd48/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:15,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:15,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:15,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:15,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:15,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0f38e9ae28b3a3299ac12ff8d13dfd48; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68907413, jitterRate=0.02680046856403351}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:15,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0f38e9ae28b3a3299ac12ff8d13dfd48: Running coprocessor pre-open hook at 1676145255487Writing region info on filesystem at 1676145255487Initializing all the Stores at 1676145255488 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145255488Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145255488Cleaning up temporary data from old regions at 1676145255583 (+95 ms)Cleaning up detritus from prior splits at 1676145255583Running coprocessor post-open hooks at 1676145255586 (+3 ms)Region opened successfully at 1676145255586 2023-02-11 19:54:15,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48., pid=33, masterSystemTime=1676145255483 2023-02-11 19:54:15,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:15,588 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:15,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:54:15,588 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:15,588 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:15,588 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): 0f38e9ae28b3a3299ac12ff8d13dfd48/cf1 is initiating minor compaction (all files) 2023-02-11 19:54:15,588 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:15,588 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): 0f38e9ae28b3a3299ac12ff8d13dfd48/cf2 is initiating minor compaction (all files) 2023-02-11 19:54:15,588 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0f38e9ae28b3a3299ac12ff8d13dfd48/cf1 in testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:15,588 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0f38e9ae28b3a3299ac12ff8d13dfd48/cf2 in testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:15,589 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/.tmp, totalSize=34.9 K 2023-02-11 19:54:15,589 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/.tmp, totalSize=34.9 K 2023-02-11 19:54:15,589 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145250278 2023-02-11 19:54:15,589 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145250278 2023-02-11 19:54:15,590 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145250480 2023-02-11 19:54:15,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:15,590 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145250480 2023-02-11 19:54:15,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:15,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:15,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4fdf3f42835ea1d9ba8f222165d44edd, NAME => 'testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 19:54:15,590 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145250692 2023-02-11 19:54:15,590 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145250692 2023-02-11 19:54:15,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=0f38e9ae28b3a3299ac12ff8d13dfd48, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:15,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:15,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145255591"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145255591"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145255591"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145255591"}]},"ts":"1676145255591"} 2023-02-11 19:54:15,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,591 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145250899 2023-02-11 19:54:15,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:15,591 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145250899 2023-02-11 19:54:15,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,592 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145251099 2023-02-11 19:54:15,592 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145251099 2023-02-11 19:54:15,592 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145251302 2023-02-11 19:54:15,593 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145251302 2023-02-11 19:54:15,593 INFO [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,593 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145251502 2023-02-11 19:54:15,593 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145251502 2023-02-11 19:54:15,595 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1 2023-02-11 19:54:15,595 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1 2023-02-11 19:54:15,595 INFO [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fdf3f42835ea1d9ba8f222165d44edd columnFamilyName cf1 2023-02-11 19:54:15,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-11 19:54:15,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 0f38e9ae28b3a3299ac12ff8d13dfd48, server=jenkins-hbase4.apache.org,42733,1676145241889 in 263 msec 2023-02-11 19:54:15,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0f38e9ae28b3a3299ac12ff8d13dfd48, ASSIGN in 424 msec 2023-02-11 19:54:15,613 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0f38e9ae28b3a3299ac12ff8d13dfd48#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:15,613 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): 0f38e9ae28b3a3299ac12ff8d13dfd48#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:15,617 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7-top 2023-02-11 19:54:15,621 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d-top 2023-02-11 19:54:15,627 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14-top 2023-02-11 19:54:15,633 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba-top 2023-02-11 19:54:15,633 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/.tmp/cf1/67f07bcc7ec74c8b83098750addf2f20 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/67f07bcc7ec74c8b83098750addf2f20 2023-02-11 19:54:15,638 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/.tmp/cf2/367bb4f2d1744970995c5a2a04ed2c99 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/367bb4f2d1744970995c5a2a04ed2c99 2023-02-11 19:54:15,639 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b-top 2023-02-11 19:54:15,640 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0f38e9ae28b3a3299ac12ff8d13dfd48/cf1 of 0f38e9ae28b3a3299ac12ff8d13dfd48 into 67f07bcc7ec74c8b83098750addf2f20(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:15,640 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0f38e9ae28b3a3299ac12ff8d13dfd48: 2023-02-11 19:54:15,640 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48., storeName=0f38e9ae28b3a3299ac12ff8d13dfd48/cf1, priority=9, startTime=1676145255587; duration=0sec 2023-02-11 19:54:15,641 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:15,644 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec-top 2023-02-11 19:54:15,645 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0f38e9ae28b3a3299ac12ff8d13dfd48/cf2 of 0f38e9ae28b3a3299ac12ff8d13dfd48 into 367bb4f2d1744970995c5a2a04ed2c99(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:15,645 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0f38e9ae28b3a3299ac12ff8d13dfd48: 2023-02-11 19:54:15,645 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48., storeName=0f38e9ae28b3a3299ac12ff8d13dfd48/cf2, priority=9, startTime=1676145255588; duration=0sec 2023-02-11 19:54:15,645 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:15,648 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb-top 2023-02-11 19:54:15,648 INFO [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(310): Store=4fdf3f42835ea1d9ba8f222165d44edd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:15,649 INFO [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,649 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2 2023-02-11 19:54:15,649 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2 2023-02-11 19:54:15,650 INFO [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fdf3f42835ea1d9ba8f222165d44edd columnFamilyName cf2 2023-02-11 19:54:15,663 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b-top 2023-02-11 19:54:15,666 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18-top 2023-02-11 19:54:15,670 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1-top 2023-02-11 19:54:15,673 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24-top 2023-02-11 19:54:15,677 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba-top 2023-02-11 19:54:15,681 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1-top 2023-02-11 19:54:15,685 DEBUG [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb-top 2023-02-11 19:54:15,686 INFO [StoreOpener-4fdf3f42835ea1d9ba8f222165d44edd-1] regionserver.HStore(310): Store=4fdf3f42835ea1d9ba8f222165d44edd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:15,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:15,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:15,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4fdf3f42835ea1d9ba8f222165d44edd; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61938428, jitterRate=-0.0770455002784729}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:15,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4fdf3f42835ea1d9ba8f222165d44edd: Running coprocessor pre-open hook at 1676145255591Writing region info on filesystem at 1676145255591Initializing all the Stores at 1676145255592 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145255592Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145255593 (+1 ms)Cleaning up temporary data from old regions at 1676145255688 (+95 ms)Cleaning up detritus from prior splits at 1676145255688Running coprocessor post-open hooks at 1676145255692 (+4 ms)Region opened successfully at 1676145255692 2023-02-11 19:54:15,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd., pid=34, masterSystemTime=1676145255483 2023-02-11 19:54:15,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:15,694 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:15,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:54:15,694 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:15,694 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:15,694 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): 4fdf3f42835ea1d9ba8f222165d44edd/cf1 is initiating minor compaction (all files) 2023-02-11 19:54:15,694 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:15,694 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4fdf3f42835ea1d9ba8f222165d44edd/cf1 in testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:15,694 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): 4fdf3f42835ea1d9ba8f222165d44edd/cf2 is initiating minor compaction (all files) 2023-02-11 19:54:15,694 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4fdf3f42835ea1d9ba8f222165d44edd/cf2 in testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:15,695 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba-top] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/.tmp, totalSize=34.9 K 2023-02-11 19:54:15,695 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18-top] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/.tmp, totalSize=34.9 K 2023-02-11 19:54:15,696 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145250278 2023-02-11 19:54:15,696 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145250278 2023-02-11 19:54:15,696 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145250480 2023-02-11 19:54:15,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:15,697 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145250480 2023-02-11 19:54:15,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:15,697 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145250692 2023-02-11 19:54:15,697 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=4fdf3f42835ea1d9ba8f222165d44edd, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:15,697 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145255697"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145255697"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145255697"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145255697"}]},"ts":"1676145255697"} 2023-02-11 19:54:15,697 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145250692 2023-02-11 19:54:15,697 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676145250899 2023-02-11 19:54:15,698 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676145250899 2023-02-11 19:54:15,698 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676145251099 2023-02-11 19:54:15,699 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676145251099 2023-02-11 19:54:15,699 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676145251302 2023-02-11 19:54:15,699 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676145251302 2023-02-11 19:54:15,699 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145251502 2023-02-11 19:54:15,700 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145251502 2023-02-11 19:54:15,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-11 19:54:15,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 4fdf3f42835ea1d9ba8f222165d44edd, server=jenkins-hbase4.apache.org,42733,1676145241889 in 363 msec 2023-02-11 19:54:15,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-11 19:54:15,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fdf3f42835ea1d9ba8f222165d44edd, ASSIGN in 530 msec 2023-02-11 19:54:15,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd in 3.9400 sec 2023-02-11 19:54:15,717 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4fdf3f42835ea1d9ba8f222165d44edd#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:15,717 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): 4fdf3f42835ea1d9ba8f222165d44edd#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:15,735 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/.tmp/cf1/e48511da33454aadbb4cdd0c6209177f as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/e48511da33454aadbb4cdd0c6209177f 2023-02-11 19:54:15,738 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/.tmp/cf2/3eb9a1fc6fd74a148c4a3e683b541dd0 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/3eb9a1fc6fd74a148c4a3e683b541dd0 2023-02-11 19:54:15,741 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4fdf3f42835ea1d9ba8f222165d44edd/cf1 of 4fdf3f42835ea1d9ba8f222165d44edd into e48511da33454aadbb4cdd0c6209177f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:15,741 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4fdf3f42835ea1d9ba8f222165d44edd: 2023-02-11 19:54:15,741 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd., storeName=4fdf3f42835ea1d9ba8f222165d44edd/cf1, priority=9, startTime=1676145255693; duration=0sec 2023-02-11 19:54:15,742 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:15,743 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4fdf3f42835ea1d9ba8f222165d44edd/cf2 of 4fdf3f42835ea1d9ba8f222165d44edd into 3eb9a1fc6fd74a148c4a3e683b541dd0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:15,743 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4fdf3f42835ea1d9ba8f222165d44edd: 2023-02-11 19:54:15,743 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd., storeName=4fdf3f42835ea1d9ba8f222165d44edd/cf2, priority=9, startTime=1676145255694; duration=0sec 2023-02-11 19:54:15,743 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:15,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42733] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:45194 deadline: 1676145315918, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d. is not online on jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:16,030 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 19:54:16,037 INFO [Listener at localhost/38771] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 19:54:16,039 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 19:54:16,055 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:16,057 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-11 19:54:16,057 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-11 19:54:16,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:16,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 19:54:16,061 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145256061"}]},"ts":"1676145256061"} 2023-02-11 19:54:16,062 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-11 19:54:16,065 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-11 19:54:16,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0f38e9ae28b3a3299ac12ff8d13dfd48, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fdf3f42835ea1d9ba8f222165d44edd, UNASSIGN}] 2023-02-11 19:54:16,066 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fdf3f42835ea1d9ba8f222165d44edd, UNASSIGN 2023-02-11 19:54:16,067 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=4fdf3f42835ea1d9ba8f222165d44edd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:16,068 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145256067"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145256067"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145256067"}]},"ts":"1676145256067"} 2023-02-11 19:54:16,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 4fdf3f42835ea1d9ba8f222165d44edd, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:16,071 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0f38e9ae28b3a3299ac12ff8d13dfd48, UNASSIGN 2023-02-11 19:54:16,072 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=0f38e9ae28b3a3299ac12ff8d13dfd48, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:16,072 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145256072"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145256072"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145256072"}]},"ts":"1676145256072"} 2023-02-11 19:54:16,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 0f38e9ae28b3a3299ac12ff8d13dfd48, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:16,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 19:54:16,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:16,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4fdf3f42835ea1d9ba8f222165d44edd, disabling compactions & flushes 2023-02-11 19:54:16,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:16,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:16,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. after waiting 0 ms 2023-02-11 19:54:16,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:16,232 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba-top] to archive 2023-02-11 19:54:16,233 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:16,235 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,236 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,238 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,239 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,240 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,241 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,243 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,244 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18-top] to archive 2023-02-11 19:54:16,245 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:16,247 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,249 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,250 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,251 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,253 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,254 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,255 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:54:16,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:16,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd. 2023-02-11 19:54:16,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4fdf3f42835ea1d9ba8f222165d44edd: Waiting for close lock at 1676145256221Running coprocessor pre-close hooks at 1676145256221Disabling compacts and flushes for region at 1676145256221Disabling writes for close at 1676145256221Writing region close event to WAL at 1676145256256 (+35 ms)Running coprocessor post-close hooks at 1676145256259 (+3 ms)Closed at 1676145256260 (+1 ms) 2023-02-11 19:54:16,261 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:16,261 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:16,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0f38e9ae28b3a3299ac12ff8d13dfd48, disabling compactions & flushes 2023-02-11 19:54:16,261 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:16,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:16,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. after waiting 0 ms 2023-02-11 19:54:16,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:16,262 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=4fdf3f42835ea1d9ba8f222165d44edd, regionState=CLOSED 2023-02-11 19:54:16,262 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145256262"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145256262"}]},"ts":"1676145256262"} 2023-02-11 19:54:16,263 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba-bottom] to archive 2023-02-11 19:54:16,264 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:16,266 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-11 19:54:16,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 4fdf3f42835ea1d9ba8f222165d44edd, server=jenkins-hbase4.apache.org,42733,1676145241889 in 194 msec 2023-02-11 19:54:16,267 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4fdf3f42835ea1d9ba8f222165d44edd, UNASSIGN in 200 msec 2023-02-11 19:54:16,269 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,271 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,272 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,273 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,274 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,276 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18-bottom] to archive 2023-02-11 19:54:16,277 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:16,278 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,279 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,280 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,281 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,282 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,283 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,284 DEBUG [StoreCloser-testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:54:16,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:16,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48. 2023-02-11 19:54:16,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0f38e9ae28b3a3299ac12ff8d13dfd48: Waiting for close lock at 1676145256261Running coprocessor pre-close hooks at 1676145256261Disabling compacts and flushes for region at 1676145256261Disabling writes for close at 1676145256262 (+1 ms)Writing region close event to WAL at 1676145256285 (+23 ms)Running coprocessor post-close hooks at 1676145256288 (+3 ms)Closed at 1676145256288 2023-02-11 19:54:16,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:16,290 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=0f38e9ae28b3a3299ac12ff8d13dfd48, regionState=CLOSED 2023-02-11 19:54:16,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145256290"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145256290"}]},"ts":"1676145256290"} 2023-02-11 19:54:16,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-11 19:54:16,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 0f38e9ae28b3a3299ac12ff8d13dfd48, server=jenkins-hbase4.apache.org,42733,1676145241889 in 219 msec 2023-02-11 19:54:16,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-11 19:54:16,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0f38e9ae28b3a3299ac12ff8d13dfd48, UNASSIGN in 228 msec 2023-02-11 19:54:16,296 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145256296"}]},"ts":"1676145256296"} 2023-02-11 19:54:16,298 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-11 19:54:16,299 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-11 19:54:16,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 243 msec 2023-02-11 19:54:16,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 19:54:16,363 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-11 19:54:16,364 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-11 19:54:16,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:16,367 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:16,368 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:16,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-11 19:54:16,374 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,375 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:16,375 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:16,377 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/recovered.edits] 2023-02-11 19:54:16,378 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/recovered.edits] 2023-02-11 19:54:16,378 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/recovered.edits] 2023-02-11 19:54:16,382 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/42a205d1c88b498d9c9f013593e524d7 2023-02-11 19:54:16,383 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/.regioninfo to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/.regioninfo 2023-02-11 19:54:16,383 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/.regioninfo to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/.regioninfo 2023-02-11 19:54:16,384 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/47215e09f5dd47d8ae1d858013fcba7d 2023-02-11 19:54:16,386 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/6fe312ca09454c359047d18c10ebeb14 2023-02-11 19:54:16,386 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,387 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/42a205d1c88b498d9c9f013593e524d7.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,388 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/9123fe3aeef24307b6a1ca20ccf3dbba 2023-02-11 19:54:16,388 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,388 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/47215e09f5dd47d8ae1d858013fcba7d.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,390 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,390 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/6fe312ca09454c359047d18c10ebeb14.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,390 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/97846742557342ed85c83d1040a33c5b 2023-02-11 19:54:16,391 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,391 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/9123fe3aeef24307b6a1ca20ccf3dbba.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,392 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/bc946d87bea9492091d0bf464b3f37ec 2023-02-11 19:54:16,393 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,393 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf1/c20b827a189a4cc7992ef377dd1243eb 2023-02-11 19:54:16,393 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/97846742557342ed85c83d1040a33c5b.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,395 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,395 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/bc946d87bea9492091d0bf464b3f37ec.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,397 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,397 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/c20b827a189a4cc7992ef377dd1243eb.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,397 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0618e56885e7491ebfe5f05cc094fb3b 2023-02-11 19:54:16,399 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/0e876483ccc246ca8479eb1cd3e03d18 2023-02-11 19:54:16,401 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,401 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/1143961ec97c46f19aa196e0bf65e2a1 2023-02-11 19:54:16,402 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0618e56885e7491ebfe5f05cc094fb3b.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,403 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,403 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/556081219aea44fab0e30a14389b8a24 2023-02-11 19:54:16,404 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/0e876483ccc246ca8479eb1cd3e03d18.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,404 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,405 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/588e8ee6e523402c8ca95fe75beb8cba 2023-02-11 19:54:16,406 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/1143961ec97c46f19aa196e0bf65e2a1.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,406 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,407 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/8f2f3c70b49e48b88bcf29d52ef811e1 2023-02-11 19:54:16,408 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/556081219aea44fab0e30a14389b8a24.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,409 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,410 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/cf2/b41291a2e3d1472b9fd125a0732778eb 2023-02-11 19:54:16,411 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,413 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,415 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/recovered.edits/85.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d/recovered.edits/85.seqid 2023-02-11 19:54:16,415 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/588e8ee6e523402c8ca95fe75beb8cba.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,415 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,416 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/67f07bcc7ec74c8b83098750addf2f20 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf1/67f07bcc7ec74c8b83098750addf2f20 2023-02-11 19:54:16,417 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/8f2f3c70b49e48b88bcf29d52ef811e1.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,418 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/4fdf3f42835ea1d9ba8f222165d44edd/cf2/b41291a2e3d1472b9fd125a0732778eb.281aa262152bf186516a3159abcf093d 2023-02-11 19:54:16,419 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/367bb4f2d1744970995c5a2a04ed2c99 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/cf2/367bb4f2d1744970995c5a2a04ed2c99 2023-02-11 19:54:16,420 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/e48511da33454aadbb4cdd0c6209177f to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf1/e48511da33454aadbb4cdd0c6209177f 2023-02-11 19:54:16,422 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48/recovered.edits/90.seqid 2023-02-11 19:54:16,423 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/0f38e9ae28b3a3299ac12ff8d13dfd48 2023-02-11 19:54:16,424 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/3eb9a1fc6fd74a148c4a3e683b541dd0 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/cf2/3eb9a1fc6fd74a148c4a3e683b541dd0 2023-02-11 19:54:16,426 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd/recovered.edits/90.seqid 2023-02-11 19:54:16,427 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitWithoutPONR/4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:16,427 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-11 19:54:16,429 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:16,432 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-11 19:54:16,434 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-11 19:54:16,436 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:16,436 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-11 19:54:16,436 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145256436"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:16,436 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145256436"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:16,436 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145256436"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:16,438 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 19:54:16,439 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 281aa262152bf186516a3159abcf093d, NAME => 'testSplitWithoutPONR,,1676145249773.281aa262152bf186516a3159abcf093d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 0f38e9ae28b3a3299ac12ff8d13dfd48, NAME => 'testSplitWithoutPONR,,1676145251766.0f38e9ae28b3a3299ac12ff8d13dfd48.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 4fdf3f42835ea1d9ba8f222165d44edd, NAME => 'testSplitWithoutPONR,41,1676145251766.4fdf3f42835ea1d9ba8f222165d44edd.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 19:54:16,439 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-11 19:54:16,439 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145256439"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:16,442 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-11 19:54:16,444 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:16,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 80 msec 2023-02-11 19:54:16,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-11 19:54:16,469 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-11 19:54:16,484 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=395 (was 390) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33854 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:39624 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:39438 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5876c91-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:39166 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:39654 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:60558 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:58972 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:58982 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33770 [Waiting for operation #39] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33894 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:60624 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36654 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:60620 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33968 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36688 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5876c91-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:39454 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=852 (was 832) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=401 (was 437), ProcessCount=168 (was 168), AvailableMemoryMB=3784 (was 3878) 2023-02-11 19:54:16,498 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=395, OpenFileDescriptor=852, MaxFileDescriptor=60000, SystemLoadAverage=401, ProcessCount=168, AvailableMemoryMB=3783 2023-02-11 19:54:16,498 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:16,501 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:54:16,501 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:54:16,503 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:16,504 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:54:16,505 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:54:16,506 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,507 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c empty. 2023-02-11 19:54:16,507 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,507 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-11 19:54:16,518 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:16,519 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => b15981c436d314b9d2350f98152f693c, NAME => 'testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:54:16,528 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:16,528 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing b15981c436d314b9d2350f98152f693c, disabling compactions & flushes 2023-02-11 19:54:16,528 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:16,528 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:16,528 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. after waiting 0 ms 2023-02-11 19:54:16,529 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:16,529 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:16,529 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for b15981c436d314b9d2350f98152f693c: Waiting for close lock at 1676145256528Disabling compacts and flushes for region at 1676145256528Disabling writes for close at 1676145256529 (+1 ms)Writing region close event to WAL at 1676145256529Closed at 1676145256529 2023-02-11 19:54:16,531 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:54:16,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145256531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145256531"}]},"ts":"1676145256531"} 2023-02-11 19:54:16,533 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:54:16,534 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:54:16,534 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145256534"}]},"ts":"1676145256534"} 2023-02-11 19:54:16,535 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-11 19:54:16,538 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:16,539 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:16,539 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:16,539 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:16,539 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:54:16,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN}] 2023-02-11 19:54:16,541 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:16,542 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43747,1676145241852; forceNewPlan=false, retain=false 2023-02-11 19:54:16,692 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:16,693 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:16,693 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145256693"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145256693"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145256693"}]},"ts":"1676145256693"} 2023-02-11 19:54:16,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:16,847 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:16,848 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 19:54:16,851 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38378, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 19:54:16,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:16,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b15981c436d314b9d2350f98152f693c, NAME => 'testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:16,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:16,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:16,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,861 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,862 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1 2023-02-11 19:54:16,862 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1 2023-02-11 19:54:16,863 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b15981c436d314b9d2350f98152f693c columnFamilyName cf1 2023-02-11 19:54:16,863 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(310): Store=b15981c436d314b9d2350f98152f693c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:16,863 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,865 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2 2023-02-11 19:54:16,865 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2 2023-02-11 19:54:16,865 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b15981c436d314b9d2350f98152f693c columnFamilyName cf2 2023-02-11 19:54:16,866 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(310): Store=b15981c436d314b9d2350f98152f693c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:16,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:16,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:16,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:16,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b15981c436d314b9d2350f98152f693c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60265404, jitterRate=-0.10197550058364868}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:16,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b15981c436d314b9d2350f98152f693c: Running coprocessor pre-open hook at 1676145256859Writing region info on filesystem at 1676145256859Initializing all the Stores at 1676145256860 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145256860Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145256860Cleaning up temporary data from old regions at 1676145256868 (+8 ms)Cleaning up detritus from prior splits at 1676145256868Running coprocessor post-open hooks at 1676145256873 (+5 ms)Region opened successfully at 1676145256873 2023-02-11 19:54:16,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c., pid=43, masterSystemTime=1676145256847 2023-02-11 19:54:16,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:16,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:16,878 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:16,879 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145256878"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145256878"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145256878"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145256878"}]},"ts":"1676145256878"} 2023-02-11 19:54:16,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-11 19:54:16,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 in 185 msec 2023-02-11 19:54:16,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-11 19:54:16,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN in 343 msec 2023-02-11 19:54:16,885 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:54:16,885 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145256885"}]},"ts":"1676145256885"} 2023-02-11 19:54:16,886 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-11 19:54:16,892 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:54:16,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 390 msec 2023-02-11 19:54:17,006 DEBUG [Listener at localhost/38771] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 19:54:17,010 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38392, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 19:54:17,017 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:17,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 19:54:17,019 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,019 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,019 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 19:54:17,019 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:54:17,019 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,019 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,021 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,021 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,021 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,021 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,022 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,022 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,022 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,023 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,023 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,025 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,025 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,025 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,025 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,026 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,026 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,028 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:17,028 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,028 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:17,028 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. started... 2023-02-11 19:54:17,029 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b15981c436d314b9d2350f98152f693c 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 19:54:17,044 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/4e8a3782de54488cb9850fb62bf942df 2023-02-11 19:54:17,066 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/1515405ee38340c8b985cbc86a6cab06 2023-02-11 19:54:17,071 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/4e8a3782de54488cb9850fb62bf942df as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df 2023-02-11 19:54:17,075 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:54:17,076 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/1515405ee38340c8b985cbc86a6cab06 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06 2023-02-11 19:54:17,080 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:54:17,081 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b15981c436d314b9d2350f98152f693c in 52ms, sequenceid=6, compaction requested=false 2023-02-11 19:54:17,081 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,081 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b15981c436d314b9d2350f98152f693c: 2023-02-11 19:54:17,081 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,081 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:17,081 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:17,081 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,082 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,082 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,084 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,084 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,085 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,085 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 19:54:17,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45d57295[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:17,087 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:54:17,087 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,088 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:17,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,088 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,088 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,088 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,088 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,088 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,088 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,089 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:17,089 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:17,089 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:17,090 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,090 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,090 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,090 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,090 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,090 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,091 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,091 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,091 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,091 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:54:17,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:54:17,095 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 19:54:17,095 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:17,095 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:17,095 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 19:54:17,095 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:17,097 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,097 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,097 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,097 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,097 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,097 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,098 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,098 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,098 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,099 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,101 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,101 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,101 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,105 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,105 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:17,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,105 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:17,105 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:17,206 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:17,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:17,222 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:17,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:17,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 19:54:17,224 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,224 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,224 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 19:54:17,224 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:54:17,224 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,224 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,226 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,226 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,226 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,226 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,226 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,226 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:17,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,229 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,229 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,229 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,229 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,229 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,230 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,231 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,231 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,231 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,231 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,231 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:17,231 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:17,231 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,231 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. started... 2023-02-11 19:54:17,232 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b15981c436d314b9d2350f98152f693c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:17,236 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,236 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,236 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,236 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,236 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,246 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/d491f3c221de46c19be8954586b7f2c0 2023-02-11 19:54:17,260 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/0a30a2121da44f4394fe516c89f9f0b9 2023-02-11 19:54:17,264 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/d491f3c221de46c19be8954586b7f2c0 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0 2023-02-11 19:54:17,269 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:54:17,269 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/0a30a2121da44f4394fe516c89f9f0b9 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9 2023-02-11 19:54:17,275 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:54:17,277 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b15981c436d314b9d2350f98152f693c in 45ms, sequenceid=19, compaction requested=false 2023-02-11 19:54:17,277 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b15981c436d314b9d2350f98152f693c: 2023-02-11 19:54:17,277 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,277 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:17,277 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:17,277 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,277 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,277 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,279 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,279 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,279 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,279 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 19:54:17,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@584fbed6[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:17,282 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:54:17,282 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,283 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,283 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,283 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,284 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,284 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,284 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,284 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,284 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:17,284 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,284 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:17,284 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:17,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,284 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,285 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,285 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:17,286 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,286 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,286 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,291 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,291 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,291 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,291 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,291 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,291 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,291 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:54:17,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:54:17,295 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 19:54:17,295 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:17,295 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:17,295 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 19:54:17,295 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:17,297 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,297 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,298 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,297 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,299 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,299 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,302 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,302 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,303 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,306 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,306 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,306 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,307 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,306 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,307 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,307 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,307 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,307 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,307 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:17,307 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,307 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,308 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:17,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,307 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,308 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,308 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:17,308 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,308 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,308 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,308 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,309 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,309 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,408 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:17,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:17,433 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:17,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:17,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 19:54:17,435 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,435 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,435 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 19:54:17,435 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:54:17,435 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,435 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,437 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,437 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,437 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,437 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,437 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,437 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:17,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,438 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:17,439 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:17,439 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,439 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. started... 2023-02-11 19:54:17,439 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b15981c436d314b9d2350f98152f693c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:17,440 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,440 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,441 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,441 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,441 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,441 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,449 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/a175561522dd4288be4d83e3bef54a48 2023-02-11 19:54:17,466 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/9bae3e4194b94e12a9550f4522a0182e 2023-02-11 19:54:17,470 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/a175561522dd4288be4d83e3bef54a48 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48 2023-02-11 19:54:17,474 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:54:17,475 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/9bae3e4194b94e12a9550f4522a0182e as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e 2023-02-11 19:54:17,479 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:54:17,479 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b15981c436d314b9d2350f98152f693c in 40ms, sequenceid=32, compaction requested=true 2023-02-11 19:54:17,480 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b15981c436d314b9d2350f98152f693c: 2023-02-11 19:54:17,480 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,480 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:17,480 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:17,480 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,480 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,480 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,482 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,482 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,482 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,482 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 19:54:17,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d39995f[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:17,484 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:54:17,485 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,487 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:17,487 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:17,487 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:17,489 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,489 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,489 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,490 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,491 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:54:17,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:54:17,493 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 19:54:17,493 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:17,493 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:17,493 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 19:54:17,493 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:17,496 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,496 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,496 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,496 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,496 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,496 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,497 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,497 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,498 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,498 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,501 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,507 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,507 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,507 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,508 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,608 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:17,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:17,625 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:17,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:17,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 19:54:17,627 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,627 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,627 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 19:54:17,627 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:54:17,627 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,627 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,630 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,630 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,630 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,630 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,630 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,630 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,632 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,632 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,632 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,632 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,632 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,632 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,632 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,632 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,633 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,634 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,634 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:17,638 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,638 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. started... 2023-02-11 19:54:17,638 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b15981c436d314b9d2350f98152f693c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:17,639 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,640 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,640 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,650 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/da3eb61bea114517b63b828894c107b8 2023-02-11 19:54:17,665 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/45165b396e12421ab7ec32239aefca91 2023-02-11 19:54:17,671 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/da3eb61bea114517b63b828894c107b8 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8 2023-02-11 19:54:17,675 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:54:17,676 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/45165b396e12421ab7ec32239aefca91 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91 2023-02-11 19:54:17,680 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:54:17,681 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b15981c436d314b9d2350f98152f693c in 43ms, sequenceid=45, compaction requested=true 2023-02-11 19:54:17,681 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b15981c436d314b9d2350f98152f693c: 2023-02-11 19:54:17,681 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,681 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:17,681 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:17,681 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,681 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,681 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,683 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,683 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,684 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,684 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 19:54:17,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54855557[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:17,686 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:54:17,686 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,688 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,688 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,688 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,688 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:17,688 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,688 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,689 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,689 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,689 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:17,689 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,689 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:17,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,689 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,689 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,689 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:17,691 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:54:17,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:54:17,694 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 19:54:17,694 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:17,694 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:17,694 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 19:54:17,695 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:17,696 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,696 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,696 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,696 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,696 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,696 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,697 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,696 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,697 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,698 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,699 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,700 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,700 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,704 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,704 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,705 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,704 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,705 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,805 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:17,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:17,821 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:17,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:17,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 19:54:17,823 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,823 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,823 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 19:54:17,823 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:54:17,823 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,823 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,824 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,824 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,824 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,824 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,825 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,825 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:17,827 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,827 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. started... 2023-02-11 19:54:17,827 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b15981c436d314b9d2350f98152f693c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:17,828 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,829 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,829 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,829 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,829 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,829 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,849 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/b956104cd71c4cc189c7f37a802ffd0a 2023-02-11 19:54:17,862 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/febf7109914b440c99ef0608d885e234 2023-02-11 19:54:17,867 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/b956104cd71c4cc189c7f37a802ffd0a as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a 2023-02-11 19:54:17,871 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:54:17,871 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/febf7109914b440c99ef0608d885e234 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234 2023-02-11 19:54:17,875 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:54:17,876 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b15981c436d314b9d2350f98152f693c in 49ms, sequenceid=58, compaction requested=true 2023-02-11 19:54:17,876 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b15981c436d314b9d2350f98152f693c: 2023-02-11 19:54:17,876 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:17,876 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:17,876 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:17,876 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,876 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:17,876 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:17,879 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,879 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,879 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,879 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:17,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 19:54:17,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3aaac488[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:17,881 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:54:17,881 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,882 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,882 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,882 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,882 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,882 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,883 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:17,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,883 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,883 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,883 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,883 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:17,883 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:17,883 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:17,883 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:17,883 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:17,884 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,884 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,884 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,884 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,888 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,888 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,888 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:17,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,889 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:17,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:54:17,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:54:17,891 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 19:54:17,891 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:17,892 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:17,892 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 19:54:17,892 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:17,893 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:17,893 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:17,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,893 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,894 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,894 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,894 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:17,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:17,896 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,897 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,897 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,900 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,900 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,900 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,900 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:17,900 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,900 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:17,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,901 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,901 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,900 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 19:54:17,901 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:17,900 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:17,900 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:17,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,901 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:17,901 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,901 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:17,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,901 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:17,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:17,901 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,902 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:17,902 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:17,902 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:17,902 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:17,902 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,001 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:18,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:18,017 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:18,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:18,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 19:54:18,018 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 19:54:18,018 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:18,018 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 19:54:18,018 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:54:18,019 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,019 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:18,020 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:18,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:18,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:18,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:18,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:18,020 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,020 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:18,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,021 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:18,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 19:54:18,022 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:18,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:18,022 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:18,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:18,022 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:18,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:18,022 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:18,023 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:18,023 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:18,023 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. started... 2023-02-11 19:54:18,023 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b15981c436d314b9d2350f98152f693c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:18,024 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,024 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,024 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,024 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:18,024 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,024 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:18,032 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/f59ccadd411a48fba53a0496e0dd63a4 2023-02-11 19:54:18,043 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/fc84ef2b845a4113ace6537b9583ef0b 2023-02-11 19:54:18,048 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/f59ccadd411a48fba53a0496e0dd63a4 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4 2023-02-11 19:54:18,053 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:54:18,054 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/fc84ef2b845a4113ace6537b9583ef0b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b 2023-02-11 19:54:18,057 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:54:18,058 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b15981c436d314b9d2350f98152f693c in 35ms, sequenceid=71, compaction requested=true 2023-02-11 19:54:18,058 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b15981c436d314b9d2350f98152f693c: 2023-02-11 19:54:18,058 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:18,058 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:18,058 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:18,058 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,059 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 19:54:18,059 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 19:54:18,060 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,060 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:18,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:18,061 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,061 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:18,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:18,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:18,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:18,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:18,063 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 19:54:18,063 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fb67405[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:18,063 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:54:18,063 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,065 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,065 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,065 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:18,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:18,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,066 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,066 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 19:54:18,066 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:18,067 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:18,067 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:18,067 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:18,068 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:18,068 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 19:54:18,068 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:18,068 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,068 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:18,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:18,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:18,068 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:18,068 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 19:54:18,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:18,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:18,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:18,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:18,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:18,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:54:18,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:54:18,071 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 19:54:18,071 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:18,071 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:18,071 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 19:54:18,071 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:18,073 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:18,073 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:18,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:18,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:18,073 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:18,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:18,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:18,073 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:18,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:18,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:18,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:18,074 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:18,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:18,074 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:18,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:18,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,076 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:18,076 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 19:54:18,076 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:18,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:18,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:18,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:18,082 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 19:54:18,082 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:18,082 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:18,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:18,082 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:18,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:18,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:18,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 19:54:18,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:18,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:18,083 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 19:54:18,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:18,183 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:18,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:18,444 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 19:54:18,445 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:18,446 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,448 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:18,448 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:18,449 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:18,697 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,697 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:18,697 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:18,697 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:18,697 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-11 19:54:18,697 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:18,697 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:18,697 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:18,697 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:18,698 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:18,699 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 19:54:18,700 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:18,700 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:18,700 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:18,700 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:18,701 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:18,701 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:18,701 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:18,702 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:18,702 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:18,702 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:18,702 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:18,702 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:18,703 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:18,703 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,703 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:18,703 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,703 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,703 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,703 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:18,704 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:18,704 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:18,705 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145288705 2023-02-11 19:54:18,706 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:18,706 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,708 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:18,709 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:18,709 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:18,709 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145318709 2023-02-11 19:54:18,709 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145378709 2023-02-11 19:54:18,709 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:18,709 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:18,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:18,711 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:18,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:18,959 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,959 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:18,959 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:18,959 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:18,961 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:18,961 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:18,961 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:18,961 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:18,961 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:18,962 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:18,963 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 19:54:18,963 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:18,963 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:18,964 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:18,964 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:18,964 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:18,964 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:18,965 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:18,965 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:18,965 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:18,965 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:18,965 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:18,966 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:18,966 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:18,966 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,966 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:18,966 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,966 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,966 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,966 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:18,967 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:18,967 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:18,968 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145288968 2023-02-11 19:54:18,968 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:18,969 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,972 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:18,972 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:18,972 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:18,972 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145318972 2023-02-11 19:54:18,972 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145378972 2023-02-11 19:54:18,972 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:54:18,973 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:18,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN}] 2023-02-11 19:54:18,974 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:18,974 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:18,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:18,974 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:19,223 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,223 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:19,223 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:19,223 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:19,224 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:19,224 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:19,224 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:19,224 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:19,224 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:19,225 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:19,226 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 19:54:19,226 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:19,227 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:19,227 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:19,227 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:19,227 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:19,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:19,229 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,229 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,230 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:19,230 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,230 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:19,230 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,231 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,231 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,231 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:19,231 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:19,232 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:19,233 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145289233 2023-02-11 19:54:19,235 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,235 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:19,237 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:19,238 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:19,238 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:19,238 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145319238 2023-02-11 19:54:19,238 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145379238 2023-02-11 19:54:19,238 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:54:19,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN}] 2023-02-11 19:54:19,239 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:19,241 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:19,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:19,243 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145259242"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145259242"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145259242"}]},"ts":"1676145259242"} 2023-02-11 19:54:19,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:19,245 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:19,245 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:19,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:19,245 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:19,488 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,488 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:19,489 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:19,489 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:19,491 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:19,492 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:19,492 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:19,492 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:19,492 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:19,493 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:19,494 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:19,494 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 19:54:19,494 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:19,494 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:19,495 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:19,495 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:19,495 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,495 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:19,495 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,496 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,496 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,496 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:19,496 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,496 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,497 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:19,497 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,497 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:19,498 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:19,498 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,498 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,498 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,498 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:19,498 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:19,498 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:19,499 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:19,499 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:19,499 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c to restore RIT 2023-02-11 19:54:19,499 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145289499 2023-02-11 19:54:19,499 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:19,500 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:19,502 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:19,503 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:19,503 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:19,503 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145319503 2023-02-11 19:54:19,503 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145379503 2023-02-11 19:54:19,503 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:19,503 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:19,503 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145259503"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145259503"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145259503"}]},"ts":"1676145259503"} 2023-02-11 19:54:19,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:19,505 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:19,659 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:19,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b15981c436d314b9d2350f98152f693c, disabling compactions & flushes 2023-02-11 19:54:19,659 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:19,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:19,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. after waiting 0 ms 2023-02-11 19:54:19,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:19,659 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing b15981c436d314b9d2350f98152f693c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 19:54:19,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/af38ba351a184a1d802e4f9405539adf 2023-02-11 19:54:19,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/64e94dc61fcf40319c56288730141747 2023-02-11 19:54:19,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/af38ba351a184a1d802e4f9405539adf as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf 2023-02-11 19:54:19,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:54:19,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/64e94dc61fcf40319c56288730141747 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747 2023-02-11 19:54:19,697 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:54:19,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b15981c436d314b9d2350f98152f693c in 39ms, sequenceid=82, compaction requested=true 2023-02-11 19:54:19,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 19:54:19,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:19,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:19,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b15981c436d314b9d2350f98152f693c: Waiting for close lock at 1676145259659Running coprocessor pre-close hooks at 1676145259659Disabling compacts and flushes for region at 1676145259659Disabling writes for close at 1676145259659Obtaining lock to block concurrent updates at 1676145259659Preparing flush snapshotting stores in b15981c436d314b9d2350f98152f693c at 1676145259659Finished memstore snapshotting testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676145259659Flushing stores of testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. at 1676145259660 (+1 ms)Flushing b15981c436d314b9d2350f98152f693c/cf1: creating writer at 1676145259661 (+1 ms)Flushing b15981c436d314b9d2350f98152f693c/cf1: appending metadata at 1676145259663 (+2 ms)Flushing b15981c436d314b9d2350f98152f693c/cf1: closing flushed file at 1676145259663Flushing b15981c436d314b9d2350f98152f693c/cf2: creating writer at 1676145259675 (+12 ms)Flushing b15981c436d314b9d2350f98152f693c/cf2: appending metadata at 1676145259676 (+1 ms)Flushing b15981c436d314b9d2350f98152f693c/cf2: closing flushed file at 1676145259676Flushing b15981c436d314b9d2350f98152f693c/cf1: reopening flushed file at 1676145259689 (+13 ms)Flushing b15981c436d314b9d2350f98152f693c/cf2: reopening flushed file at 1676145259694 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b15981c436d314b9d2350f98152f693c in 39ms, sequenceid=82, compaction requested=true at 1676145259698 (+4 ms)Writing region close event to WAL at 1676145259700 (+2 ms)Running coprocessor post-close hooks at 1676145259703 (+3 ms)Closed at 1676145259704 (+1 ms) 2023-02-11 19:54:19,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:19,706 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=CLOSED 2023-02-11 19:54:19,707 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145259706"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145259706"}]},"ts":"1676145259706"} 2023-02-11 19:54:19,708 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:19,708 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:19,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:19,708 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:19,753 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,753 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:19,753 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:19,753 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:19,754 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:19,754 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:19,754 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:19,754 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:19,754 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:19,754 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:19,755 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:19,755 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 19:54:19,755 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-11 19:54:19,756 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:19,756 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:19,756 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:19,756 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:19,757 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,757 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:19,757 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,757 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:19,757 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,758 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:19,758 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,758 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:19,758 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:19,759 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,759 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:19,759 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:19,759 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:19,759 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,759 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,759 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:19,759 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:19,759 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:19,759 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:19,759 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:19,760 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:19,760 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:19,761 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c to restore RIT 2023-02-11 19:54:19,761 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145289761 2023-02-11 19:54:19,762 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:19,762 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:19,766 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:19,766 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:19,766 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:19,766 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145319766 2023-02-11 19:54:19,766 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145379766 2023-02-11 19:54:19,766 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:54:19,767 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:19,767 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145259767"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145259767"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145259767"}]},"ts":"1676145259767"} 2023-02-11 19:54:19,768 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:19,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-11 19:54:19,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 in 263 msec 2023-02-11 19:54:19,771 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:19,771 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:19,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:19,771 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:20,017 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,017 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:20,017 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:20,017 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:20,018 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:20,018 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:20,018 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:20,018 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:20,018 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:20,019 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:20,020 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:20,020 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 19:54:20,020 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:20,021 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:20,021 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:20,022 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:20,022 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:20,022 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,023 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:20,023 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,023 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,023 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,024 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:20,024 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,024 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,024 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:20,025 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,025 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:20,026 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:20,026 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:20,026 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,026 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,026 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,026 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:20,026 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:20,026 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 19:54:20,027 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:20,027 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:20,028 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c to restore RIT 2023-02-11 19:54:20,028 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145290028 2023-02-11 19:54:20,029 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:20,029 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:20,031 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:20,031 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:20,031 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:20,031 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145320031 2023-02-11 19:54:20,031 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145380031 2023-02-11 19:54:20,031 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:54:20,031 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:20,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:20,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-11 19:54:20,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN in 795 msec 2023-02-11 19:54:20,039 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:20,039 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:20,039 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:20,281 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,282 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:20,282 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:20,282 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:20,283 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:20,283 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:20,283 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:20,283 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:20,283 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:20,284 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:20,285 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:20,285 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:20,285 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:20,286 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:20,286 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:20,286 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:20,287 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:20,287 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,287 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:20,288 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,288 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,288 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,289 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:20,289 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,289 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,289 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:20,290 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,290 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:20,290 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:20,291 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:20,291 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:20,291 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:20,291 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 19:54:20,291 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:20,292 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:20,292 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145290292 2023-02-11 19:54:20,294 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:20,294 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,296 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:20,296 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:20,296 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:20,296 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145320296 2023-02-11 19:54:20,296 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145380296 2023-02-11 19:54:20,296 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:54:20,296 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,296 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,296 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,302 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:20,311 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=b15981c436d314b9d2350f98152f693c, threads=14 2023-02-11 19:54:20,312 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,312 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,313 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,313 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,313 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,313 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,314 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,314 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,316 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,317 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,317 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,317 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,318 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,318 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,505 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,520 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,520 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,529 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,531 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,531 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,536 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,552 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,556 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,560 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,561 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,579 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,579 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,586 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,586 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region b15981c436d314b9d2350f98152f693c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:54:20,630 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:20,630 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:20,631 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:20,797 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,797 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:20,797 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:20,797 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:20,798 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:20,798 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:20,798 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:20,798 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:20,799 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:20,799 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:20,800 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 19:54:20,801 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:20,801 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:20,801 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:20,802 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:20,802 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:20,802 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:20,802 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:20,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:20,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:20,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:20,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:20,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:20,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:20,805 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:20,805 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,805 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,805 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,805 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:20,806 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:20,806 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 19:54:20,806 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:20,807 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:20,810 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145290810 2023-02-11 19:54:20,812 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:20,812 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:20,814 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:20,814 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:20,814 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:20,814 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145320814 2023-02-11 19:54:20,815 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145380815 2023-02-11 19:54:20,815 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:20,827 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=b15981c436d314b9d2350f98152f693c, threads=14 2023-02-11 19:54:20,828 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,828 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,828 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,828 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,829 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,829 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,829 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,829 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,829 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,829 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,829 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,830 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,830 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,830 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:20,989 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 19:54:21,036 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,044 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,048 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,048 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,048 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,060 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,060 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,065 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,066 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,074 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,074 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,077 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,091 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,094 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06 for region: b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:21,094 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region b15981c436d314b9d2350f98152f693c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:54:21,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:21,150 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/7161492ed949257357e0dcca7cfffe17/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:54:21,153 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/5a05c3396f111fa56b9bac0f72076513/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:54:21,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:21,153 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:21,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:21,315 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,315 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:21,315 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:21,315 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:21,316 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:21,317 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:21,317 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:21,317 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:21,317 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:21,318 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:21,318 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 19:54:21,319 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:21,319 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:21,319 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:21,319 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:21,319 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:21,320 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:21,320 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,320 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:21,320 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,320 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,321 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,321 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:21,321 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,321 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,321 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:21,322 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,322 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:21,322 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:21,324 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:21,324 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,324 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,324 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,324 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:21,324 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:21,325 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 19:54:21,325 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:21,325 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:21,328 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145291328 2023-02-11 19:54:21,329 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,329 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:21,331 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:21,331 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:21,331 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:21,331 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145321331 2023-02-11 19:54:21,331 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145381331 2023-02-11 19:54:21,332 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:54:21,334 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:21,337 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:21,337 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:21,337 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:21,582 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,582 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:21,582 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:21,583 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:21,584 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:21,584 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:21,584 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:21,584 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:21,584 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:21,584 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:21,585 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 19:54:21,585 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:21,585 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:21,586 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:21,586 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:21,586 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:21,586 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:21,587 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,587 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:21,587 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,587 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,587 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,588 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:21,588 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,588 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,588 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:21,589 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,589 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:21,589 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:21,589 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:21,589 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,589 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,589 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,589 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:21,589 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:21,589 DEBUG [Listener at localhost/38771] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,589 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:21,590 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:21,590 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:21,591 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145291591 2023-02-11 19:54:21,593 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,593 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:21,595 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:21,596 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:21,596 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:21,596 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145321596 2023-02-11 19:54:21,596 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145381596 2023-02-11 19:54:21,596 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:54:21,596 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:21,608 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:21,608 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 19:54:21,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:21,608 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:21,847 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,847 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:21,847 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:21,847 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:21,847 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:21,848 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:21,848 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:21,848 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:21,848 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:21,848 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:21,849 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-11 19:54:21,850 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:21,850 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:21,850 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:21,850 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:21,850 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:21,850 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:21,851 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,851 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:21,851 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,851 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:21,852 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,852 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:21,852 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,852 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:21,853 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:21,853 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,853 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:21,854 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:21,854 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:21,854 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,854 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,854 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,854 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:21,854 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:21,854 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:21,855 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:21,855 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:21,855 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145291855 2023-02-11 19:54:21,857 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:21,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:21,860 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:21,860 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:21,860 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:21,861 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145321861 2023-02-11 19:54:21,861 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145381861 2023-02-11 19:54:21,861 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:21,861 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:21,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:21,863 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 19:54:21,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:21,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:22,112 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,112 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:22,112 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:22,112 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:22,113 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:22,113 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:22,113 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:22,113 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:22,113 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:22,114 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:22,115 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 19:54:22,115 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:22,115 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:22,115 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:22,115 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:22,115 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:22,115 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:22,116 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,116 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:22,116 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,116 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,117 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,117 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:22,117 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,117 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,117 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:22,118 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,118 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:22,118 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:22,118 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:22,118 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,119 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,119 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,119 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:22,119 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:22,119 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:22,119 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:22,120 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:22,120 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145292120 2023-02-11 19:54:22,121 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:22,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,123 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:22,124 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:22,124 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:22,124 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145322124 2023-02-11 19:54:22,124 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145382124 2023-02-11 19:54:22,124 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:54:22,124 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:22,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:22,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 19:54:22,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:22,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:22,374 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,374 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:22,374 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:22,374 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:22,375 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:22,376 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:22,376 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:22,376 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:22,376 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:22,376 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:22,377 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 19:54:22,377 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:22,377 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:22,378 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:22,378 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:22,378 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:22,378 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:22,379 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,379 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:22,379 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,379 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,379 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,380 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:22,380 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,380 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,380 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:22,381 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,381 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:22,381 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:22,381 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:22,381 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,381 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,381 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,381 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:22,381 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:22,381 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:22,382 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:22,382 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:22,383 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145292383 2023-02-11 19:54:22,383 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:22,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,386 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:22,386 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:22,387 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:22,387 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145322387 2023-02-11 19:54:22,387 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145382387 2023-02-11 19:54:22,387 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:54:22,387 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:22,391 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN] 2023-02-11 19:54:22,391 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:22,391 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 19:54:22,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:22,391 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:22,637 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,637 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:22,637 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:22,637 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:22,640 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:22,640 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:22,640 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:22,640 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:22,640 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:22,641 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:22,641 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 19:54:22,641 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:54:22,641 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:54:22,642 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 19:54:22,642 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:22,642 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:22,642 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:22,642 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:22,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:22,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:22,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:22,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:22,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:22,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:22,645 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:22,645 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,645 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,646 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,646 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:22,646 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:22,646 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:22,646 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:22,646 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:22,647 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:22,647 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c to restore RIT 2023-02-11 19:54:22,647 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145292647 2023-02-11 19:54:22,647 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:22,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,650 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:22,651 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:22,652 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:22,652 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:22,652 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145322652 2023-02-11 19:54:22,652 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145382652 2023-02-11 19:54:22,652 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:54:22,652 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:22,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:22,657 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 19:54:22,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:22,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:22,902 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,902 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:22,902 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:22,902 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:22,903 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:22,903 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:22,903 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:22,903 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:22,903 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:22,903 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:22,904 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:54:22,904 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 19:54:22,905 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:22,905 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:22,905 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:22,905 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:22,905 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,906 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:22,906 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,906 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:22,906 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,906 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:22,906 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,907 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:22,907 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:22,907 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,907 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:22,907 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:22,908 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,908 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,908 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,908 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:22,908 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:22,908 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:22,909 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:22,909 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c to restore RIT 2023-02-11 19:54:22,909 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145292909 2023-02-11 19:54:22,909 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:22,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:22,912 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:22,914 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:22,914 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:22,914 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:22,914 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145322914 2023-02-11 19:54:22,914 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145382914 2023-02-11 19:54:22,914 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:54:22,914 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:22,917 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:22,917 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 19:54:22,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:22,917 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:23,164 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,165 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:23,165 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:23,165 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:23,165 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:23,165 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:23,165 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:23,165 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:23,165 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:23,166 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:23,167 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 19:54:23,167 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 19:54:23,167 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:23,167 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:23,167 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:23,167 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:23,168 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,168 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:23,168 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,168 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,168 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,169 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:23,169 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,169 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,169 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:23,169 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,169 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,169 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:23,170 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,170 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,170 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,170 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:23,170 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-11 19:54:23,170 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:23,171 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:23,171 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c to restore RIT 2023-02-11 19:54:23,171 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145293171 2023-02-11 19:54:23,172 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:23,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,174 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,176 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:23,176 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:23,176 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:23,176 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145323176 2023-02-11 19:54:23,176 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145383176 2023-02-11 19:54:23,176 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 19:54:23,177 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:23,179 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:23,179 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 19:54:23,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:23,179 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:23,427 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,427 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:23,427 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:23,427 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:23,427 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:23,427 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:23,427 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:23,427 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:23,427 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:23,428 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:23,429 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 19:54:23,429 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 19:54:23,429 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:23,429 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:23,429 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:23,430 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:23,430 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,430 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:23,430 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,431 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,431 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,431 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:23,431 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,432 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,432 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:23,432 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,433 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,433 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:23,433 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,433 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,433 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,433 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:54:23,433 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 19:54:23,434 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:23,434 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:23,435 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c to restore RIT 2023-02-11 19:54:23,435 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145293435 2023-02-11 19:54:23,435 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:23,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,439 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,441 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:23,441 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:23,441 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:23,441 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145323441 2023-02-11 19:54:23,441 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145383441 2023-02-11 19:54:23,441 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-11 19:54:23,441 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:54:23,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 exec-time=4.9960 sec 2023-02-11 19:54:23,444 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,444 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43747,1676145241852; forceNewPlan=false, retain=true 2023-02-11 19:54:23,595 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:23,596 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:23,596 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145263596"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145263596"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145263596"}]},"ts":"1676145263596"} 2023-02-11 19:54:23,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:23,598 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:54:23,598 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:54:23,598 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:54:23,691 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:23,691 WARN [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-11 19:54:23,691 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:54:23,691 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:54:23,692 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:54:23,692 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:54:23,692 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:54:23,692 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:54:23,692 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:54:23,692 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:54:23,693 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:54:23,694 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-11 19:54:23,694 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:54:23,694 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:54:23,695 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:54:23,695 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:54:23,695 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,695 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:54:23,696 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,696 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:54:23,696 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,697 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:54:23,697 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,697 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:54:23,697 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:23,697 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:54:23,698 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,698 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:54:23,698 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,698 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 19:54:23,698 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:54:23,699 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:54:23,699 INFO [Listener at localhost/38771] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c to restore RIT 2023-02-11 19:54:23,699 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145293699 2023-02-11 19:54:23,700 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:54:23,700 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:54:23,702 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:54:23,702 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b15981c436d314b9d2350f98152f693c, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:54:23,702 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:54:23,703 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145323703 2023-02-11 19:54:23,703 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145383703 2023-02-11 19:54:23,703 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:54:23,703 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:23,703 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145263703"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145263703"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145263703"}]},"ts":"1676145263703"} 2023-02-11 19:54:23,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:23,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:23,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b15981c436d314b9d2350f98152f693c, NAME => 'testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:23,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:23,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:23,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:23,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:23,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:23,863 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:23,864 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1 2023-02-11 19:54:23,864 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1 2023-02-11 19:54:23,864 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b15981c436d314b9d2350f98152f693c columnFamilyName cf1 2023-02-11 19:54:23,871 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df 2023-02-11 19:54:23,874 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48 2023-02-11 19:54:23,877 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf 2023-02-11 19:54:23,880 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a 2023-02-11 19:54:23,882 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0 2023-02-11 19:54:23,885 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8 2023-02-11 19:54:23,888 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4 2023-02-11 19:54:23,888 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(310): Store=b15981c436d314b9d2350f98152f693c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:23,889 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:23,889 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2 2023-02-11 19:54:23,889 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2 2023-02-11 19:54:23,890 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b15981c436d314b9d2350f98152f693c columnFamilyName cf2 2023-02-11 19:54:23,896 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9 2023-02-11 19:54:23,899 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06 2023-02-11 19:54:23,903 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91 2023-02-11 19:54:23,906 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747 2023-02-11 19:54:23,909 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e 2023-02-11 19:54:23,912 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b 2023-02-11 19:54:23,915 DEBUG [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234 2023-02-11 19:54:23,915 INFO [StoreOpener-b15981c436d314b9d2350f98152f693c-1] regionserver.HStore(310): Store=b15981c436d314b9d2350f98152f693c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:23,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:23,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:23,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.splits 2023-02-11 19:54:23,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:23,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:23,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b15981c436d314b9d2350f98152f693c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68416679, jitterRate=0.019487962126731873}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:23,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b15981c436d314b9d2350f98152f693c: Running coprocessor pre-open hook at 1676145263862Writing region info on filesystem at 1676145263862Initializing all the Stores at 1676145263862Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145263862Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145263863 (+1 ms)Cleaning up temporary data from old regions at 1676145263917 (+54 ms)Cleaning up detritus from prior splits at 1676145263918 (+1 ms)Running coprocessor post-open hooks at 1676145263921 (+3 ms)Region opened successfully at 1676145263921 2023-02-11 19:54:23,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c., pid=48, masterSystemTime=1676145263857 2023-02-11 19:54:23,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:23,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:54:23,923 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:23,923 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:23,924 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 19:54:23,924 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 19:54:23,925 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1912): b15981c436d314b9d2350f98152f693c/cf2 is initiating minor compaction (all files) 2023-02-11 19:54:23,925 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1912): b15981c436d314b9d2350f98152f693c/cf1 is initiating minor compaction (all files) 2023-02-11 19:54:23,925 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b15981c436d314b9d2350f98152f693c/cf2 in testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:23,925 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b15981c436d314b9d2350f98152f693c/cf1 in testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:23,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:23,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:23,925 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp, totalSize=34.9 K 2023-02-11 19:54:23,925 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp, totalSize=34.9 K 2023-02-11 19:54:23,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:23,926 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 1515405ee38340c8b985cbc86a6cab06, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145257010 2023-02-11 19:54:23,926 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145263926"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145263926"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145263926"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145263926"}]},"ts":"1676145263926"} 2023-02-11 19:54:23,926 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 4e8a3782de54488cb9850fb62bf942df, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145257010 2023-02-11 19:54:23,926 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 0a30a2121da44f4394fe516c89f9f0b9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145257208 2023-02-11 19:54:23,926 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting d491f3c221de46c19be8954586b7f2c0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145257208 2023-02-11 19:54:23,926 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 9bae3e4194b94e12a9550f4522a0182e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145257410 2023-02-11 19:54:23,927 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting a175561522dd4288be4d83e3bef54a48, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145257410 2023-02-11 19:54:23,927 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 45165b396e12421ab7ec32239aefca91, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145257610 2023-02-11 19:54:23,927 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting da3eb61bea114517b63b828894c107b8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145257610 2023-02-11 19:54:23,927 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting febf7109914b440c99ef0608d885e234, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145257807 2023-02-11 19:54:23,928 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting b956104cd71c4cc189c7f37a802ffd0a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145257807 2023-02-11 19:54:23,928 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting fc84ef2b845a4113ace6537b9583ef0b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145258003 2023-02-11 19:54:23,928 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting f59ccadd411a48fba53a0496e0dd63a4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145258003 2023-02-11 19:54:23,928 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 64e94dc61fcf40319c56288730141747, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145258185 2023-02-11 19:54:23,928 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting af38ba351a184a1d802e4f9405539adf, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145258185 2023-02-11 19:54:23,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-11 19:54:23,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 in 222 msec 2023-02-11 19:54:23,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN in 1.5410 sec 2023-02-11 19:54:23,953 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] throttle.PressureAwareThroughputController(145): b15981c436d314b9d2350f98152f693c#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:23,954 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] throttle.PressureAwareThroughputController(145): b15981c436d314b9d2350f98152f693c#cf1#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:23,973 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf2/722380a1234f46019a3a18c2fa4a7240 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/722380a1234f46019a3a18c2fa4a7240 2023-02-11 19:54:23,977 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/.tmp/cf1/9beb21eacbf243e1b752cbb554c0b778 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/9beb21eacbf243e1b752cbb554c0b778 2023-02-11 19:54:23,980 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b15981c436d314b9d2350f98152f693c/cf2 of b15981c436d314b9d2350f98152f693c into 722380a1234f46019a3a18c2fa4a7240(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:23,980 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b15981c436d314b9d2350f98152f693c: 2023-02-11 19:54:23,980 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c., storeName=b15981c436d314b9d2350f98152f693c/cf2, priority=9, startTime=1676145263922; duration=0sec 2023-02-11 19:54:23,982 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:23,985 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b15981c436d314b9d2350f98152f693c/cf1 of b15981c436d314b9d2350f98152f693c into 9beb21eacbf243e1b752cbb554c0b778(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:23,985 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b15981c436d314b9d2350f98152f693c: 2023-02-11 19:54:23,985 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c., storeName=b15981c436d314b9d2350f98152f693c/cf1, priority=9, startTime=1676145263922; duration=0sec 2023-02-11 19:54:23,985 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:24,280 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-11 19:54:24,282 DEBUG [Listener at localhost/38771] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-11 19:54:24,284 INFO [Listener at localhost/38771] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-11 19:54:24,286 INFO [Listener at localhost/38771] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-11 19:54:24,287 INFO [Listener at localhost/38771] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-11 19:54:24,287 INFO [Listener at localhost/38771] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-11 19:54:24,302 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:24,304 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-11 19:54:24,304 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-11 19:54:24,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:24,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 19:54:24,308 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145264308"}]},"ts":"1676145264308"} 2023-02-11 19:54:24,309 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-11 19:54:24,311 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-11 19:54:24,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN}] 2023-02-11 19:54:24,312 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN 2023-02-11 19:54:24,312 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:24,313 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145264312"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145264312"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145264312"}]},"ts":"1676145264312"} 2023-02-11 19:54:24,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:24,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 19:54:24,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:24,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b15981c436d314b9d2350f98152f693c, disabling compactions & flushes 2023-02-11 19:54:24,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:24,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:24,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. after waiting 1 ms 2023-02-11 19:54:24,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:24,466 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf] to archive 2023-02-11 19:54:24,467 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:24,469 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/4e8a3782de54488cb9850fb62bf942df 2023-02-11 19:54:24,470 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/d491f3c221de46c19be8954586b7f2c0 2023-02-11 19:54:24,471 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/a175561522dd4288be4d83e3bef54a48 2023-02-11 19:54:24,472 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/da3eb61bea114517b63b828894c107b8 2023-02-11 19:54:24,473 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/b956104cd71c4cc189c7f37a802ffd0a 2023-02-11 19:54:24,474 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/f59ccadd411a48fba53a0496e0dd63a4 2023-02-11 19:54:24,475 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/af38ba351a184a1d802e4f9405539adf 2023-02-11 19:54:24,480 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747] to archive 2023-02-11 19:54:24,480 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:24,481 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/1515405ee38340c8b985cbc86a6cab06 2023-02-11 19:54:24,482 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/0a30a2121da44f4394fe516c89f9f0b9 2023-02-11 19:54:24,483 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/9bae3e4194b94e12a9550f4522a0182e 2023-02-11 19:54:24,484 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/45165b396e12421ab7ec32239aefca91 2023-02-11 19:54:24,485 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/febf7109914b440c99ef0608d885e234 2023-02-11 19:54:24,486 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/fc84ef2b845a4113ace6537b9583ef0b 2023-02-11 19:54:24,487 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/64e94dc61fcf40319c56288730141747 2023-02-11 19:54:24,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:54:24,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:24,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c. 2023-02-11 19:54:24,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b15981c436d314b9d2350f98152f693c: Waiting for close lock at 1676145264465Running coprocessor pre-close hooks at 1676145264465Disabling compacts and flushes for region at 1676145264465Disabling writes for close at 1676145264466 (+1 ms)Writing region close event to WAL at 1676145264488 (+22 ms)Running coprocessor post-close hooks at 1676145264492 (+4 ms)Closed at 1676145264492 2023-02-11 19:54:24,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:24,494 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=b15981c436d314b9d2350f98152f693c, regionState=CLOSED 2023-02-11 19:54:24,494 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145264494"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145264494"}]},"ts":"1676145264494"} 2023-02-11 19:54:24,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-11 19:54:24,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure b15981c436d314b9d2350f98152f693c, server=jenkins-hbase4.apache.org,43747,1676145241852 in 181 msec 2023-02-11 19:54:24,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-11 19:54:24,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, UNASSIGN in 186 msec 2023-02-11 19:54:24,499 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145264499"}]},"ts":"1676145264499"} 2023-02-11 19:54:24,500 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-11 19:54:24,502 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-11 19:54:24,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 199 msec 2023-02-11 19:54:24,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 19:54:24,610 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-11 19:54:24,611 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-11 19:54:24,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:24,614 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:24,614 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:24,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-11 19:54:24,618 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:24,620 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/recovered.edits] 2023-02-11 19:54:24,622 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/9beb21eacbf243e1b752cbb554c0b778 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf1/9beb21eacbf243e1b752cbb554c0b778 2023-02-11 19:54:24,624 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/722380a1234f46019a3a18c2fa4a7240 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/cf2/722380a1234f46019a3a18c2fa4a7240 2023-02-11 19:54:24,627 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c/recovered.edits/90.seqid 2023-02-11 19:54:24,628 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackAndDoubleExecution/b15981c436d314b9d2350f98152f693c 2023-02-11 19:54:24,628 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-11 19:54:24,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:24,633 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-11 19:54:24,634 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-11 19:54:24,635 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:24,635 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-11 19:54:24,635 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145264635"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:24,637 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 19:54:24,637 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b15981c436d314b9d2350f98152f693c, NAME => 'testRollbackAndDoubleExecution,,1676145256502.b15981c436d314b9d2350f98152f693c.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 19:54:24,637 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-11 19:54:24,637 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145264637"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:24,638 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-11 19:54:24,640 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:54:24,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 29 msec 2023-02-11 19:54:24,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-11 19:54:24,716 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-11 19:54:24,732 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=399 (was 395) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:39624 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:44240 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:39438 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:44186 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:43747-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:36390 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:58982 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:33770 [Waiting for operation #53] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:36374 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:33894 [Waiting for operation #45] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:44234 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:36654 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:60620 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:33968 [Waiting for operation #36] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:36688 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:53366 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1007784249_15 at /127.0.0.1:36380 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=846 (was 852), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=377 (was 401), ProcessCount=168 (was 168), AvailableMemoryMB=3706 (was 3783) 2023-02-11 19:54:24,746 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=399, OpenFileDescriptor=846, MaxFileDescriptor=60000, SystemLoadAverage=377, ProcessCount=168, AvailableMemoryMB=3705 2023-02-11 19:54:24,746 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:24,748 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:54:24,748 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:54:24,750 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:54:24,751 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:54:24,752 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:54:24,753 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:24,754 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736 empty. 2023-02-11 19:54:24,754 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:24,754 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-11 19:54:24,764 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:24,765 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => b5f5623d65d35fccbaee614c8f714736, NAME => 'testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:54:24,779 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:24,779 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing b5f5623d65d35fccbaee614c8f714736, disabling compactions & flushes 2023-02-11 19:54:24,779 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:24,779 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:24,779 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. after waiting 0 ms 2023-02-11 19:54:24,779 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:24,779 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:24,779 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for b5f5623d65d35fccbaee614c8f714736: Waiting for close lock at 1676145264779Disabling compacts and flushes for region at 1676145264779Disabling writes for close at 1676145264779Writing region close event to WAL at 1676145264779Closed at 1676145264779 2023-02-11 19:54:24,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:54:24,783 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676145264783"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145264783"}]},"ts":"1676145264783"} 2023-02-11 19:54:24,784 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:54:24,785 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:54:24,786 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145264785"}]},"ts":"1676145264785"} 2023-02-11 19:54:24,787 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-11 19:54:24,791 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:24,792 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:24,792 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:24,792 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:24,792 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:54:24,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b5f5623d65d35fccbaee614c8f714736, ASSIGN}] 2023-02-11 19:54:24,794 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b5f5623d65d35fccbaee614c8f714736, ASSIGN 2023-02-11 19:54:24,794 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b5f5623d65d35fccbaee614c8f714736, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43747,1676145241852; forceNewPlan=false, retain=false 2023-02-11 19:54:24,944 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:24,945 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=b5f5623d65d35fccbaee614c8f714736, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:24,946 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676145264945"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145264945"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145264945"}]},"ts":"1676145264945"} 2023-02-11 19:54:24,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure b5f5623d65d35fccbaee614c8f714736, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:25,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:25,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b5f5623d65d35fccbaee614c8f714736, NAME => 'testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:25,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:25,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:25,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,104 INFO [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,106 DEBUG [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/cf1 2023-02-11 19:54:25,106 DEBUG [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/cf1 2023-02-11 19:54:25,106 INFO [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5f5623d65d35fccbaee614c8f714736 columnFamilyName cf1 2023-02-11 19:54:25,107 INFO [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] regionserver.HStore(310): Store=b5f5623d65d35fccbaee614c8f714736/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:25,107 INFO [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,109 DEBUG [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/cf2 2023-02-11 19:54:25,109 DEBUG [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/cf2 2023-02-11 19:54:25,109 INFO [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5f5623d65d35fccbaee614c8f714736 columnFamilyName cf2 2023-02-11 19:54:25,109 INFO [StoreOpener-b5f5623d65d35fccbaee614c8f714736-1] regionserver.HStore(310): Store=b5f5623d65d35fccbaee614c8f714736/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:25,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:25,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:25,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b5f5623d65d35fccbaee614c8f714736; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67663007, jitterRate=0.008257374167442322}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:25,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b5f5623d65d35fccbaee614c8f714736: Running coprocessor pre-open hook at 1676145265103Writing region info on filesystem at 1676145265103Initializing all the Stores at 1676145265104 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145265104Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145265104Cleaning up temporary data from old regions at 1676145265111 (+7 ms)Cleaning up detritus from prior splits at 1676145265112 (+1 ms)Running coprocessor post-open hooks at 1676145265116 (+4 ms)Region opened successfully at 1676145265116 2023-02-11 19:54:25,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736., pid=55, masterSystemTime=1676145265099 2023-02-11 19:54:25,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:25,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:25,119 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=b5f5623d65d35fccbaee614c8f714736, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:25,119 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676145265119"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145265119"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145265119"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145265119"}]},"ts":"1676145265119"} 2023-02-11 19:54:25,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-11 19:54:25,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure b5f5623d65d35fccbaee614c8f714736, server=jenkins-hbase4.apache.org,43747,1676145241852 in 173 msec 2023-02-11 19:54:25,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-11 19:54:25,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b5f5623d65d35fccbaee614c8f714736, ASSIGN in 330 msec 2023-02-11 19:54:25,124 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:54:25,124 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145265124"}]},"ts":"1676145265124"} 2023-02-11 19:54:25,125 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-11 19:54:25,128 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:54:25,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 379 msec 2023-02-11 19:54:25,251 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:25,252 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,253 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,253 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,253 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b5f5623d65d35fccbaee614c8f714736, UNASSIGN}] 2023-02-11 19:54:25,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b5f5623d65d35fccbaee614c8f714736, UNASSIGN 2023-02-11 19:54:25,256 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=b5f5623d65d35fccbaee614c8f714736, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:25,257 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676145265256"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145265256"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145265256"}]},"ts":"1676145265256"} 2023-02-11 19:54:25,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure b5f5623d65d35fccbaee614c8f714736, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:25,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b5f5623d65d35fccbaee614c8f714736, disabling compactions & flushes 2023-02-11 19:54:25,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:25,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:25,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. after waiting 0 ms 2023-02-11 19:54:25,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:25,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-11 19:54:25,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:25,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736. 2023-02-11 19:54:25,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b5f5623d65d35fccbaee614c8f714736: Waiting for close lock at 1676145265410Running coprocessor pre-close hooks at 1676145265410Disabling compacts and flushes for region at 1676145265410Disabling writes for close at 1676145265412 (+2 ms)Writing region close event to WAL at 1676145265412Running coprocessor post-close hooks at 1676145265416 (+4 ms)Closed at 1676145265416 2023-02-11 19:54:25,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:25,419 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=b5f5623d65d35fccbaee614c8f714736, regionState=CLOSED 2023-02-11 19:54:25,419 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676145265419"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145265419"}]},"ts":"1676145265419"} 2023-02-11 19:54:25,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-11 19:54:25,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure b5f5623d65d35fccbaee614c8f714736, server=jenkins-hbase4.apache.org,43747,1676145241852 in 162 msec 2023-02-11 19:54:25,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-11 19:54:25,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b5f5623d65d35fccbaee614c8f714736, UNASSIGN in 167 msec 2023-02-11 19:54:25,457 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-11 19:54:25,459 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-11 19:54:25,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676145265461"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676145265461"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676145265461"}]},"ts":"1676145265461"} 2023-02-11 19:54:25,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145265461"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145265461"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145265461"}]},"ts":"1676145265461"} 2023-02-11 19:54:25,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145265461"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145265461"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145265461"}]},"ts":"1676145265461"} 2023-02-11 19:54:25,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4bfdaf124bab3234a067d2fdb10d3655, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e5975d0156dee323074c22305f41a650, ASSIGN}] 2023-02-11 19:54:25,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e5975d0156dee323074c22305f41a650, ASSIGN 2023-02-11 19:54:25,469 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e5975d0156dee323074c22305f41a650, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43747,1676145241852; forceNewPlan=false, retain=false 2023-02-11 19:54:25,469 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4bfdaf124bab3234a067d2fdb10d3655, ASSIGN 2023-02-11 19:54:25,470 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4bfdaf124bab3234a067d2fdb10d3655, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43747,1676145241852; forceNewPlan=false, retain=false 2023-02-11 19:54:25,619 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 19:54:25,621 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=4bfdaf124bab3234a067d2fdb10d3655, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:25,621 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145265621"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145265621"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145265621"}]},"ts":"1676145265621"} 2023-02-11 19:54:25,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 4bfdaf124bab3234a067d2fdb10d3655, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:25,625 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=e5975d0156dee323074c22305f41a650, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:25,625 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145265625"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145265625"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145265625"}]},"ts":"1676145265625"} 2023-02-11 19:54:25,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure e5975d0156dee323074c22305f41a650, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:25,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655. 2023-02-11 19:54:25,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4bfdaf124bab3234a067d2fdb10d3655, NAME => 'testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 19:54:25,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:25,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:25,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:25,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:25,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:25,779 INFO [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:25,780 DEBUG [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/cf1 2023-02-11 19:54:25,780 DEBUG [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/cf1 2023-02-11 19:54:25,780 INFO [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4bfdaf124bab3234a067d2fdb10d3655 columnFamilyName cf1 2023-02-11 19:54:25,781 INFO [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] regionserver.HStore(310): Store=4bfdaf124bab3234a067d2fdb10d3655/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:25,781 INFO [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:25,782 DEBUG [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/cf2 2023-02-11 19:54:25,782 DEBUG [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/cf2 2023-02-11 19:54:25,782 INFO [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4bfdaf124bab3234a067d2fdb10d3655 columnFamilyName cf2 2023-02-11 19:54:25,782 INFO [StoreOpener-4bfdaf124bab3234a067d2fdb10d3655-1] regionserver.HStore(310): Store=4bfdaf124bab3234a067d2fdb10d3655/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:25,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:25,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:25,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:25,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:25,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4bfdaf124bab3234a067d2fdb10d3655; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62570675, jitterRate=-0.0676242858171463}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:25,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4bfdaf124bab3234a067d2fdb10d3655: Running coprocessor pre-open hook at 1676145265778Writing region info on filesystem at 1676145265778Initializing all the Stores at 1676145265778Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145265778Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145265778Cleaning up temporary data from old regions at 1676145265785 (+7 ms)Cleaning up detritus from prior splits at 1676145265785Running coprocessor post-open hooks at 1676145265788 (+3 ms)Region opened successfully at 1676145265788 2023-02-11 19:54:25,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655., pid=61, masterSystemTime=1676145265774 2023-02-11 19:54:25,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655. 2023-02-11 19:54:25,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655. 2023-02-11 19:54:25,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650. 2023-02-11 19:54:25,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e5975d0156dee323074c22305f41a650, NAME => 'testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 19:54:25,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=4bfdaf124bab3234a067d2fdb10d3655, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:25,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:25,790 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145265790"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145265790"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145265790"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145265790"}]},"ts":"1676145265790"} 2023-02-11 19:54:25,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:25,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,792 INFO [StoreOpener-e5975d0156dee323074c22305f41a650-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,793 DEBUG [StoreOpener-e5975d0156dee323074c22305f41a650-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/cf1 2023-02-11 19:54:25,794 DEBUG [StoreOpener-e5975d0156dee323074c22305f41a650-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/cf1 2023-02-11 19:54:25,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-11 19:54:25,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 4bfdaf124bab3234a067d2fdb10d3655, server=jenkins-hbase4.apache.org,43747,1676145241852 in 170 msec 2023-02-11 19:54:25,794 INFO [StoreOpener-e5975d0156dee323074c22305f41a650-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5975d0156dee323074c22305f41a650 columnFamilyName cf1 2023-02-11 19:54:25,794 INFO [StoreOpener-e5975d0156dee323074c22305f41a650-1] regionserver.HStore(310): Store=e5975d0156dee323074c22305f41a650/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:25,795 INFO [StoreOpener-e5975d0156dee323074c22305f41a650-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4bfdaf124bab3234a067d2fdb10d3655, ASSIGN in 327 msec 2023-02-11 19:54:25,796 DEBUG [StoreOpener-e5975d0156dee323074c22305f41a650-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/cf2 2023-02-11 19:54:25,796 DEBUG [StoreOpener-e5975d0156dee323074c22305f41a650-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/cf2 2023-02-11 19:54:25,796 INFO [StoreOpener-e5975d0156dee323074c22305f41a650-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5975d0156dee323074c22305f41a650 columnFamilyName cf2 2023-02-11 19:54:25,797 INFO [StoreOpener-e5975d0156dee323074c22305f41a650-1] regionserver.HStore(310): Store=e5975d0156dee323074c22305f41a650/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:25,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:25,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e5975d0156dee323074c22305f41a650 2023-02-11 19:54:25,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e5975d0156dee323074c22305f41a650; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65994486, jitterRate=-0.016605526208877563}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:25,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e5975d0156dee323074c22305f41a650: Running coprocessor pre-open hook at 1676145265791Writing region info on filesystem at 1676145265791Initializing all the Stores at 1676145265792 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145265792Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145265792Cleaning up temporary data from old regions at 1676145265799 (+7 ms)Cleaning up detritus from prior splits at 1676145265799Running coprocessor post-open hooks at 1676145265802 (+3 ms)Region opened successfully at 1676145265802 2023-02-11 19:54:25,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650., pid=62, masterSystemTime=1676145265774 2023-02-11 19:54:25,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650. 2023-02-11 19:54:25,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650. 2023-02-11 19:54:25,804 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=e5975d0156dee323074c22305f41a650, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:25,805 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145265804"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145265804"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145265804"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145265804"}]},"ts":"1676145265804"} 2023-02-11 19:54:25,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-11 19:54:25,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure e5975d0156dee323074c22305f41a650, server=jenkins-hbase4.apache.org,43747,1676145241852 in 180 msec 2023-02-11 19:54:25,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-02-11 19:54:25,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e5975d0156dee323074c22305f41a650, ASSIGN in 340 msec 2023-02-11 19:54:25,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 in 558 msec 2023-02-11 19:54:26,006 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 19:54:26,008 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:26,010 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,010 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 19:54:26,014 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145266013"}]},"ts":"1676145266013"} 2023-02-11 19:54:26,015 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-11 19:54:26,016 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-11 19:54:26,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4bfdaf124bab3234a067d2fdb10d3655, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e5975d0156dee323074c22305f41a650, UNASSIGN}] 2023-02-11 19:54:26,017 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e5975d0156dee323074c22305f41a650, UNASSIGN 2023-02-11 19:54:26,018 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=e5975d0156dee323074c22305f41a650, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:26,018 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145266018"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145266018"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145266018"}]},"ts":"1676145266018"} 2023-02-11 19:54:26,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure e5975d0156dee323074c22305f41a650, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:26,021 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4bfdaf124bab3234a067d2fdb10d3655, UNASSIGN 2023-02-11 19:54:26,022 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=4bfdaf124bab3234a067d2fdb10d3655, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:26,022 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145266022"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145266022"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145266022"}]},"ts":"1676145266022"} 2023-02-11 19:54:26,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 4bfdaf124bab3234a067d2fdb10d3655, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:54:26,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 19:54:26,171 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e5975d0156dee323074c22305f41a650 2023-02-11 19:54:26,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e5975d0156dee323074c22305f41a650, disabling compactions & flushes 2023-02-11 19:54:26,172 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650. 2023-02-11 19:54:26,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650. 2023-02-11 19:54:26,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650. after waiting 0 ms 2023-02-11 19:54:26,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650. 2023-02-11 19:54:26,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 19:54:26,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:26,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650. 2023-02-11 19:54:26,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e5975d0156dee323074c22305f41a650: Waiting for close lock at 1676145266171Running coprocessor pre-close hooks at 1676145266171Disabling compacts and flushes for region at 1676145266171Disabling writes for close at 1676145266172 (+1 ms)Writing region close event to WAL at 1676145266172Running coprocessor post-close hooks at 1676145266177 (+5 ms)Closed at 1676145266178 (+1 ms) 2023-02-11 19:54:26,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e5975d0156dee323074c22305f41a650 2023-02-11 19:54:26,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:26,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4bfdaf124bab3234a067d2fdb10d3655, disabling compactions & flushes 2023-02-11 19:54:26,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655. 2023-02-11 19:54:26,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655. 2023-02-11 19:54:26,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655. after waiting 0 ms 2023-02-11 19:54:26,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655. 2023-02-11 19:54:26,181 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=e5975d0156dee323074c22305f41a650, regionState=CLOSED 2023-02-11 19:54:26,181 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145266180"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145266180"}]},"ts":"1676145266180"} 2023-02-11 19:54:26,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-11 19:54:26,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure e5975d0156dee323074c22305f41a650, server=jenkins-hbase4.apache.org,43747,1676145241852 in 163 msec 2023-02-11 19:54:26,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e5975d0156dee323074c22305f41a650, UNASSIGN in 167 msec 2023-02-11 19:54:26,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 19:54:26,186 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:26,186 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655. 2023-02-11 19:54:26,186 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4bfdaf124bab3234a067d2fdb10d3655: Waiting for close lock at 1676145266179Running coprocessor pre-close hooks at 1676145266179Disabling compacts and flushes for region at 1676145266179Disabling writes for close at 1676145266180 (+1 ms)Writing region close event to WAL at 1676145266181 (+1 ms)Running coprocessor post-close hooks at 1676145266186 (+5 ms)Closed at 1676145266186 2023-02-11 19:54:26,187 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:26,188 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=4bfdaf124bab3234a067d2fdb10d3655, regionState=CLOSED 2023-02-11 19:54:26,188 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145266188"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145266188"}]},"ts":"1676145266188"} 2023-02-11 19:54:26,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-11 19:54:26,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 4bfdaf124bab3234a067d2fdb10d3655, server=jenkins-hbase4.apache.org,43747,1676145241852 in 166 msec 2023-02-11 19:54:26,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-11 19:54:26,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=4bfdaf124bab3234a067d2fdb10d3655, UNASSIGN in 174 msec 2023-02-11 19:54:26,193 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145266193"}]},"ts":"1676145266193"} 2023-02-11 19:54:26,194 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-11 19:54:26,197 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-11 19:54:26,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 188 msec 2023-02-11 19:54:26,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 19:54:26,315 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-11 19:54:26,316 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,319 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-11 19:54:26,320 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,325 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:26,325 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650 2023-02-11 19:54:26,325 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:26,328 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/recovered.edits] 2023-02-11 19:54:26,328 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/recovered.edits] 2023-02-11 19:54:26,328 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/recovered.edits] 2023-02-11 19:54:26,335 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/recovered.edits/7.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650/recovered.edits/7.seqid 2023-02-11 19:54:26,336 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/e5975d0156dee323074c22305f41a650 2023-02-11 19:54:26,336 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/recovered.edits/7.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655/recovered.edits/7.seqid 2023-02-11 19:54:26,337 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/recovered.edits/4.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736/recovered.edits/4.seqid 2023-02-11 19:54:26,337 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/4bfdaf124bab3234a067d2fdb10d3655 2023-02-11 19:54:26,337 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionNoStoreFile/b5f5623d65d35fccbaee614c8f714736 2023-02-11 19:54:26,338 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-11 19:54:26,340 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,342 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-11 19:54:26,343 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-11 19:54:26,344 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,344 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-11 19:54:26,344 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145266344"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:26,344 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145266344"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:26,344 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145266344"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:26,346 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 19:54:26,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b5f5623d65d35fccbaee614c8f714736, NAME => 'testSplitTableRegionNoStoreFile,,1676145264749.b5f5623d65d35fccbaee614c8f714736.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4bfdaf124bab3234a067d2fdb10d3655, NAME => 'testSplitTableRegionNoStoreFile,,1676145265251.4bfdaf124bab3234a067d2fdb10d3655.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => e5975d0156dee323074c22305f41a650, NAME => 'testSplitTableRegionNoStoreFile,41,1676145265251.e5975d0156dee323074c22305f41a650.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 19:54:26,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-11 19:54:26,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145266346"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:26,347 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-11 19:54:26,349 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:54:26,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 33 msec 2023-02-11 19:54:26,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-11 19:54:26,421 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-11 19:54:26,437 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402 (was 399) Potentially hanging thread: hconnection-0xe288499-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-374969917_15 at /127.0.0.1:33894 [Waiting for operation #46] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-374969917_15 at /127.0.0.1:44234 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=844 (was 846), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=377 (was 377), ProcessCount=168 (was 168), AvailableMemoryMB=3704 (was 3705) 2023-02-11 19:54:26,451 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=402, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=377, ProcessCount=168, AvailableMemoryMB=3703 2023-02-11 19:54:26,451 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:26,455 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:54:26,455 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:54:26,457 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,458 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:54:26,459 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:54:26,460 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,461 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe empty. 2023-02-11 19:54:26,461 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,461 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-11 19:54:26,487 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:26,488 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8e9f8bd96b098a3d9b38f3d7bf6db2fe, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:54:26,501 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:26,501 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 8e9f8bd96b098a3d9b38f3d7bf6db2fe, disabling compactions & flushes 2023-02-11 19:54:26,501 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:26,501 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:26,501 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. after waiting 0 ms 2023-02-11 19:54:26,501 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:26,501 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:26,501 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: Waiting for close lock at 1676145266501Disabling compacts and flushes for region at 1676145266501Disabling writes for close at 1676145266501Writing region close event to WAL at 1676145266501Closed at 1676145266501 2023-02-11 19:54:26,504 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:54:26,504 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676145266504"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145266504"}]},"ts":"1676145266504"} 2023-02-11 19:54:26,506 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:54:26,507 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:54:26,507 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145266507"}]},"ts":"1676145266507"} 2023-02-11 19:54:26,508 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-11 19:54:26,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:26,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:26,512 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:26,512 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:26,512 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:54:26,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8e9f8bd96b098a3d9b38f3d7bf6db2fe, ASSIGN}] 2023-02-11 19:54:26,513 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8e9f8bd96b098a3d9b38f3d7bf6db2fe, ASSIGN 2023-02-11 19:54:26,514 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8e9f8bd96b098a3d9b38f3d7bf6db2fe, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:26,664 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:26,665 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=8e9f8bd96b098a3d9b38f3d7bf6db2fe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:26,665 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676145266665"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145266665"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145266665"}]},"ts":"1676145266665"} 2023-02-11 19:54:26,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 8e9f8bd96b098a3d9b38f3d7bf6db2fe, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:26,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:26,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8e9f8bd96b098a3d9b38f3d7bf6db2fe, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:26,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:26,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:26,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,828 INFO [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,829 DEBUG [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1 2023-02-11 19:54:26,830 DEBUG [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1 2023-02-11 19:54:26,830 INFO [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8e9f8bd96b098a3d9b38f3d7bf6db2fe columnFamilyName cf1 2023-02-11 19:54:26,831 INFO [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] regionserver.HStore(310): Store=8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:26,831 INFO [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,832 DEBUG [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2 2023-02-11 19:54:26,832 DEBUG [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2 2023-02-11 19:54:26,832 INFO [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8e9f8bd96b098a3d9b38f3d7bf6db2fe columnFamilyName cf2 2023-02-11 19:54:26,833 INFO [StoreOpener-8e9f8bd96b098a3d9b38f3d7bf6db2fe-1] regionserver.HStore(310): Store=8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:26,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:26,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:26,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:26,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8e9f8bd96b098a3d9b38f3d7bf6db2fe; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69957035, jitterRate=0.04244105517864227}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:26,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: Running coprocessor pre-open hook at 1676145266827Writing region info on filesystem at 1676145266827Initializing all the Stores at 1676145266828 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145266828Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145266828Cleaning up temporary data from old regions at 1676145266834 (+6 ms)Cleaning up detritus from prior splits at 1676145266835 (+1 ms)Running coprocessor post-open hooks at 1676145266839 (+4 ms)Region opened successfully at 1676145266839 2023-02-11 19:54:26,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe., pid=71, masterSystemTime=1676145266820 2023-02-11 19:54:26,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:26,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:26,842 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=8e9f8bd96b098a3d9b38f3d7bf6db2fe, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:26,842 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676145266842"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145266842"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145266842"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145266842"}]},"ts":"1676145266842"} 2023-02-11 19:54:26,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-11 19:54:26,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 8e9f8bd96b098a3d9b38f3d7bf6db2fe, server=jenkins-hbase4.apache.org,42733,1676145241889 in 176 msec 2023-02-11 19:54:26,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-11 19:54:26,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8e9f8bd96b098a3d9b38f3d7bf6db2fe, ASSIGN in 333 msec 2023-02-11 19:54:26,847 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:54:26,847 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145266847"}]},"ts":"1676145266847"} 2023-02-11 19:54:26,848 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-11 19:54:26,851 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:54:26,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 395 msec 2023-02-11 19:54:26,964 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:26,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,966 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:26,966 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:26,966 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:26,966 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:26,966 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,966 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,968 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:26,968 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:26,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:26,968 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:26,968 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:26,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:26,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:26,968 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:26,968 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,969 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:26,970 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:26,970 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:26,970 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:26,971 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 19:54:26,972 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,973 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,973 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,973 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:26,973 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:26,973 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:26,981 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/796b56dfcab34d22bb2864c7f5291410 2023-02-11 19:54:26,993 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/b944ceb15a4248808a1044e45f277431 2023-02-11 19:54:26,996 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/796b56dfcab34d22bb2864c7f5291410 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410 2023-02-11 19:54:26,999 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:54:27,000 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/b944ceb15a4248808a1044e45f277431 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431 2023-02-11 19:54:27,003 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:54:27,004 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 33ms, sequenceid=6, compaction requested=false 2023-02-11 19:54:27,004 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,004 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:27,004 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,004 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,008 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,008 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,010 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,010 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e54f5cd[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:27,010 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:27,010 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:27,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,012 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:27,012 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,012 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:27,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,013 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,013 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,014 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,014 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,014 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,014 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,014 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,014 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,014 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:27,017 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:27,017 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:27,017 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:27,017 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:27,017 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,017 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:27,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,020 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,021 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,021 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,023 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,023 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,023 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,029 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,029 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,030 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:27,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:27,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,130 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:27,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:27,144 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:27,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:27,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,146 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,146 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,146 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:27,146 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:27,146 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,146 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,148 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,148 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,148 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,148 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,148 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,148 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:27,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:27,150 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:27,150 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,150 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:27,150 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:27,152 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,153 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,153 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,153 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,153 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,153 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,159 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/215594f0d75d44b8946e075b872883ef 2023-02-11 19:54:27,170 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/798f30525b9e45df8a66d6f2bc4eae95 2023-02-11 19:54:27,175 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/215594f0d75d44b8946e075b872883ef as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef 2023-02-11 19:54:27,179 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:54:27,180 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/798f30525b9e45df8a66d6f2bc4eae95 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95 2023-02-11 19:54:27,184 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:54:27,185 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 35ms, sequenceid=19, compaction requested=false 2023-02-11 19:54:27,185 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:27,185 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,185 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:27,185 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:27,185 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,185 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,186 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,187 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,187 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,188 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,188 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@700526bd[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:27,189 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:27,189 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,190 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:27,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,191 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,191 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:27,191 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:27,193 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,193 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,193 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,194 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,194 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,194 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,194 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,194 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,194 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,194 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:27,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:27,197 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:27,197 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:27,197 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:27,197 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,197 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:27,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,198 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,199 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,200 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,200 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,202 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,202 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,202 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,205 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,205 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,205 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:27,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,205 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:27,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,206 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,306 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:27,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:27,321 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:27,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:27,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,323 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:27,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:27,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,326 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,326 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,326 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,326 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,326 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,326 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,329 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,329 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,329 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,332 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:27,332 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:27,332 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,332 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,332 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,332 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,332 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,332 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:27,332 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,332 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:27,333 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,334 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,334 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,343 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/bf508356a55043f281e5a0309d340624 2023-02-11 19:54:27,359 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7 2023-02-11 19:54:27,364 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/bf508356a55043f281e5a0309d340624 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624 2023-02-11 19:54:27,369 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:54:27,370 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7 2023-02-11 19:54:27,374 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:54:27,375 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 43ms, sequenceid=32, compaction requested=true 2023-02-11 19:54:27,375 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:27,375 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,375 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:27,375 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:27,375 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,375 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,375 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,381 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,381 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,381 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,381 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:27,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ce732b1[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:27,383 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:27,383 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,385 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,385 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:27,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:27,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:27,385 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:27,386 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,386 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,386 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,386 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,387 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,387 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,387 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,387 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,387 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,387 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:27,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:27,390 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:27,390 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:27,390 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:27,390 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,390 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:27,392 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,393 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,392 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,393 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,394 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,395 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,395 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,396 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,398 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,398 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,398 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:27,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,399 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:27,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,398 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,399 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,499 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:27,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:27,512 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:27,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:27,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,514 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:27,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:27,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,515 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,515 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,516 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,516 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,516 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,516 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,517 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,517 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,517 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,517 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,518 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:27,518 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,518 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:27,519 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,521 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,521 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,521 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,521 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,521 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,530 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/760c76560eee4eefbb87b85f847cdab7 2023-02-11 19:54:27,543 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/283a092c4f01419fa02d00e88c1a0973 2023-02-11 19:54:27,547 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/760c76560eee4eefbb87b85f847cdab7 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/760c76560eee4eefbb87b85f847cdab7 2023-02-11 19:54:27,550 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/760c76560eee4eefbb87b85f847cdab7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:54:27,551 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/283a092c4f01419fa02d00e88c1a0973 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/283a092c4f01419fa02d00e88c1a0973 2023-02-11 19:54:27,554 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/283a092c4f01419fa02d00e88c1a0973, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:54:27,555 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 37ms, sequenceid=45, compaction requested=true 2023-02-11 19:54:27,555 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:27,555 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,556 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:27,556 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:27,556 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,556 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,556 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,557 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,557 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,557 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,557 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:27,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30dd6b74[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:27,559 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:27,559 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,560 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,560 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,560 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,561 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,561 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:27,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:27,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,561 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:27,562 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,562 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,562 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,564 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,564 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,564 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,564 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,564 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,564 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,564 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:27,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:27,567 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:27,567 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:27,567 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:27,567 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,567 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:27,572 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,572 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,572 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,573 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,573 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,574 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,575 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,575 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,578 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,578 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:27,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,578 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,579 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:27,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,578 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,579 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,579 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:27,579 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,579 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,579 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,679 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:27,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:27,692 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:27,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:27,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,694 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,694 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,694 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:27,694 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:27,694 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,694 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,695 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,695 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,695 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,695 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,696 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,696 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,698 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,698 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,698 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,698 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,698 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,698 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,699 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,700 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,700 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:27,700 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,700 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:27,700 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:27,709 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/184d877ecfb14d38b74901982f651776 2023-02-11 19:54:27,720 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f 2023-02-11 19:54:27,724 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/184d877ecfb14d38b74901982f651776 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/184d877ecfb14d38b74901982f651776 2023-02-11 19:54:27,727 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/184d877ecfb14d38b74901982f651776, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:54:27,728 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f 2023-02-11 19:54:27,731 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:54:27,732 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 31ms, sequenceid=58, compaction requested=true 2023-02-11 19:54:27,732 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:27,732 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,732 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:27,732 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:27,732 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,732 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,732 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,734 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,734 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,735 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,735 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:27,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@190e1a3f[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:27,736 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:27,736 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,737 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,737 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,737 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,738 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,738 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:27,739 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,739 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,739 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,740 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,740 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:27,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:27,743 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:27,743 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:27,743 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:27,743 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,743 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:27,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,745 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,746 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,746 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,748 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,749 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,750 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,750 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,753 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,753 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,754 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,753 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,754 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,754 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,754 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:27,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,754 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,754 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,754 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,754 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,854 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:27,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:27,868 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:27,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:27,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,870 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,870 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,870 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:27,870 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:27,870 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,870 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,874 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,874 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,874 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,874 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,875 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,875 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,876 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,877 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,877 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:27,877 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:27,877 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,877 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,877 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:54:27,879 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,879 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,879 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,880 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,881 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,881 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,891 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/0d4a64a4f45e48da920b3f56a260c645 2023-02-11 19:54:27,909 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/601b759029df437583c54c4428864f3b 2023-02-11 19:54:27,913 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/0d4a64a4f45e48da920b3f56a260c645 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/0d4a64a4f45e48da920b3f56a260c645 2023-02-11 19:54:27,917 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/0d4a64a4f45e48da920b3f56a260c645, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:54:27,918 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/601b759029df437583c54c4428864f3b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/601b759029df437583c54c4428864f3b 2023-02-11 19:54:27,922 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/601b759029df437583c54c4428864f3b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:54:27,927 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 50ms, sequenceid=71, compaction requested=true 2023-02-11 19:54:27,927 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:27,927 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:27,927 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:27,927 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:27,927 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,927 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:27,927 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:27,929 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,929 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,929 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,929 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:27,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40e7f76d[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:27,931 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:27,931 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,932 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,932 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,932 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,933 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,933 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:27,933 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:27,936 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,936 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,936 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,936 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,936 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,936 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,937 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:27,937 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,937 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,937 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:27,940 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:27,940 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:27,940 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:27,940 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,940 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:27,941 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,941 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,941 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,941 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:27,941 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,941 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,942 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,942 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:27,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,942 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,943 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:27,944 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,944 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,944 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,946 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:27,947 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:27,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,947 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:27,947 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:27,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:27,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:27,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:27,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:27,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,947 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:27,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:27,948 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,948 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:27,948 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:27,948 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:27,948 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,048 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:28,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:28,062 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:28,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:28,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,064 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:28,064 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,064 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:28,064 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:28,064 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,064 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,065 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,065 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,065 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,065 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,066 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,066 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:28,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,067 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,067 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,067 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,067 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,068 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,067 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,068 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:28,068 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:28,068 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,068 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:28,068 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-11 19:54:28,069 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,069 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,069 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,071 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,071 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,071 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,080 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/6d232db1d45a442e9660bbae9fdf45a1 2023-02-11 19:54:28,084 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d232db1d45a442e9660bbae9fdf45a1 2023-02-11 19:54:28,092 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/b338f1037252481f97f19d11733f2a42 2023-02-11 19:54:28,095 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b338f1037252481f97f19d11733f2a42 2023-02-11 19:54:28,095 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/6d232db1d45a442e9660bbae9fdf45a1 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1 2023-02-11 19:54:28,098 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d232db1d45a442e9660bbae9fdf45a1 2023-02-11 19:54:28,098 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1, entries=9, sequenceid=83, filesize=5.1 K 2023-02-11 19:54:28,099 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/b338f1037252481f97f19d11733f2a42 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42 2023-02-11 19:54:28,102 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b338f1037252481f97f19d11733f2a42 2023-02-11 19:54:28,102 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42, entries=9, sequenceid=83, filesize=5.1 K 2023-02-11 19:54:28,103 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 35ms, sequenceid=83, compaction requested=true 2023-02-11 19:54:28,103 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:28,103 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,106 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,106 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,107 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,107 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6530377a[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:28,108 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:28,108 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,110 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,110 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:28,120 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,120 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,120 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,120 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,120 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,120 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,121 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,121 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,121 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,121 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:28,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:28,124 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:28,124 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:28,124 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:28,124 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,124 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:28,126 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,126 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,126 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,127 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,127 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,128 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,129 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,129 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,131 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:28,132 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:28,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,131 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,131 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,132 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,132 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:28,132 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,132 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:28,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,132 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,132 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,132 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,132 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,232 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:28,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:28,241 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:28,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:28,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,242 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:28,242 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,242 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:28,242 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:28,243 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,243 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,245 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,245 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,245 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,245 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,245 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,245 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:28,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,247 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,247 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,247 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,247 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,247 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:28,247 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-11 19:54:28,248 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,248 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,249 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,249 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,249 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,249 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,256 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:28,259 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:28,266 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:28,269 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:28,270 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/5c80d8af86af4abe9b56c21af43fe00f as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:28,273 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:28,273 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/5c80d8af86af4abe9b56c21af43fe00f, entries=5, sequenceid=91, filesize=5.0 K 2023-02-11 19:54:28,274 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:28,278 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:28,278 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd, entries=5, sequenceid=91, filesize=5.0 K 2023-02-11 19:54:28,279 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 32ms, sequenceid=91, compaction requested=true 2023-02-11 19:54:28,279 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:28,279 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,279 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:28,279 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:28,279 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,279 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,279 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,280 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,280 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:28,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3eed6663[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:28,282 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:28,282 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,283 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,283 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,284 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,283 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,284 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:28,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,286 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,286 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:28,289 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:28,289 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:28,289 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:28,289 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,289 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:28,290 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,290 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,290 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,290 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,290 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,291 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,291 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,292 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,293 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,293 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,294 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,298 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,298 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:28,298 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:28,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:28,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:28,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,298 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,299 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,299 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,299 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,299 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,299 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,299 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,299 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,399 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:28,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:28,407 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:54:28,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:54:28,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,408 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:28,408 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,408 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 19:54:28,408 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 19:54:28,409 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,409 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,411 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,411 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,411 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,411 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,411 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,411 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:54:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,412 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,412 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,412 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 19:54:28,412 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,412 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,412 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:54:28,413 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:54:28,413 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,413 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. started... 2023-02-11 19:54:28,413 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-11 19:54:28,414 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,414 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,414 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,414 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,422 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:28,425 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:28,433 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:28,436 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:28,437 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/4da00d7082974619ad360f2de4544f6e as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:28,440 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:28,440 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/4da00d7082974619ad360f2de4544f6e, entries=5, sequenceid=99, filesize=5.0 K 2023-02-11 19:54:28,441 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:28,444 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:28,445 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5, entries=5, sequenceid=99, filesize=5.0 K 2023-02-11 19:54:28,445 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 32ms, sequenceid=99, compaction requested=true 2023-02-11 19:54:28,445 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: 2023-02-11 19:54:28,445 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,445 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:54:28,445 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:54:28,446 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,446 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 19:54:28,446 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 19:54:28,447 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,447 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,447 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,447 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:54:28,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 19:54:28,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57db4c63[Count = 0] remaining members to acquire global barrier 2023-02-11 19:54:28,449 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 19:54:28,449 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,451 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,451 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,451 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,451 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,451 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,451 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:54:28,451 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,451 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,452 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:54:28,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 19:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 19:54:28,452 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:54:28,453 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,453 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,453 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 19:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,454 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,454 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 19:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:54:28,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:54:28,457 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 19:54:28,457 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:54:28,457 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:54:28,457 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,457 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:54:28,458 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,458 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,458 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:54:28,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,458 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,459 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:54:28,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,459 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,459 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,461 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:54:28,461 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,461 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,465 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:54:28,465 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 19:54:28,465 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:54:28,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:54:28,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:54:28,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:54:28,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,465 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,466 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:54:28,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:54:28,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:54:28,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:28,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:54:28,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:54:28,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:28,566 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:54:28,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:54:28,569 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,570 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:28,571 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:28,571 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:28,571 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:28,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8e9f8bd96b098a3d9b38f3d7bf6db2fe, UNASSIGN}] 2023-02-11 19:54:28,573 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8e9f8bd96b098a3d9b38f3d7bf6db2fe, UNASSIGN 2023-02-11 19:54:28,574 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=8e9f8bd96b098a3d9b38f3d7bf6db2fe, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:28,574 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676145268574"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145268574"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145268574"}]},"ts":"1676145268574"} 2023-02-11 19:54:28,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 8e9f8bd96b098a3d9b38f3d7bf6db2fe, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:28,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8e9f8bd96b098a3d9b38f3d7bf6db2fe, disabling compactions & flushes 2023-02-11 19:54:28,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. after waiting 0 ms 2023-02-11 19:54:28,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-11 19:54:28,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:28,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:28,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:28,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:28,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf1/6c7a44bbdaf64a3fbbc220212959a979 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:28,756 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:28,756 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6c7a44bbdaf64a3fbbc220212959a979, entries=1, sequenceid=103, filesize=4.9 K 2023-02-11 19:54:28,756 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.tmp/cf2/dd6839ce97cc43d79946989382bcf84e as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:28,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:28,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/dd6839ce97cc43d79946989382bcf84e, entries=1, sequenceid=103, filesize=4.9 K 2023-02-11 19:54:28,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 33ms, sequenceid=103, compaction requested=true 2023-02-11 19:54:28,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-11 19:54:28,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:28,765 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. 2023-02-11 19:54:28,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8e9f8bd96b098a3d9b38f3d7bf6db2fe: Waiting for close lock at 1676145268727Running coprocessor pre-close hooks at 1676145268727Disabling compacts and flushes for region at 1676145268727Disabling writes for close at 1676145268727Obtaining lock to block concurrent updates at 1676145268727Preparing flush snapshotting stores in 8e9f8bd96b098a3d9b38f3d7bf6db2fe at 1676145268727Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676145268727Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe. at 1676145268727Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1: creating writer at 1676145268728 (+1 ms)Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1: appending metadata at 1676145268730 (+2 ms)Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1: closing flushed file at 1676145268730Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2: creating writer at 1676145268740 (+10 ms)Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2: appending metadata at 1676145268742 (+2 ms)Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2: closing flushed file at 1676145268742Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1: reopening flushed file at 1676145268753 (+11 ms)Flushing 8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2: reopening flushed file at 1676145268757 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 8e9f8bd96b098a3d9b38f3d7bf6db2fe in 33ms, sequenceid=103, compaction requested=true at 1676145268760 (+3 ms)Writing region close event to WAL at 1676145268762 (+2 ms)Running coprocessor post-close hooks at 1676145268765 (+3 ms)Closed at 1676145268765 2023-02-11 19:54:28,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,767 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=8e9f8bd96b098a3d9b38f3d7bf6db2fe, regionState=CLOSED 2023-02-11 19:54:28,767 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676145268767"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145268767"}]},"ts":"1676145268767"} 2023-02-11 19:54:28,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-11 19:54:28,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 8e9f8bd96b098a3d9b38f3d7bf6db2fe, server=jenkins-hbase4.apache.org,42733,1676145241889 in 193 msec 2023-02-11 19:54:28,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-11 19:54:28,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8e9f8bd96b098a3d9b38f3d7bf6db2fe, UNASSIGN in 197 msec 2023-02-11 19:54:28,785 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=8e9f8bd96b098a3d9b38f3d7bf6db2fe, threads=16 2023-02-11 19:54:28,785 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,785 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/283a092c4f01419fa02d00e88c1a0973 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,785 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/601b759029df437583c54c4428864f3b for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,786 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,786 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,786 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,786 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,786 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,787 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,787 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/0d4a64a4f45e48da920b3f56a260c645 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,787 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/dd6839ce97cc43d79946989382bcf84e for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,792 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/184d877ecfb14d38b74901982f651776 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,796 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,796 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/5c80d8af86af4abe9b56c21af43fe00f for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,796 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/4da00d7082974619ad360f2de4544f6e for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,796 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6c7a44bbdaf64a3fbbc220212959a979 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,804 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/0d4a64a4f45e48da920b3f56a260c645, top=false 2023-02-11 19:54:28,806 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:28,807 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b338f1037252481f97f19d11733f2a42 2023-02-11 19:54:28,808 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f, top=false 2023-02-11 19:54:28,809 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:28,809 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/601b759029df437583c54c4428864f3b, top=false 2023-02-11 19:54:28,810 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:28,812 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/283a092c4f01419fa02d00e88c1a0973, top=false 2023-02-11 19:54:28,814 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:28,816 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:28,816 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/4da00d7082974619ad360f2de4544f6e, top=true 2023-02-11 19:54:28,817 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/184d877ecfb14d38b74901982f651776, top=false 2023-02-11 19:54:28,818 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:28,819 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:28,819 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/dd6839ce97cc43d79946989382bcf84e, top=true 2023-02-11 19:54:28,819 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:28,821 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:28,821 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5, top=true 2023-02-11 19:54:28,824 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:28,824 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd, top=true 2023-02-11 19:54:28,825 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:28,825 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/5c80d8af86af4abe9b56c21af43fe00f, top=true 2023-02-11 19:54:28,825 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:28,825 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6c7a44bbdaf64a3fbbc220212959a979, top=true 2023-02-11 19:54:28,833 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-4da00d7082974619ad360f2de4544f6e for child: 0380d0a7ec1b73252e4007b78f1d4d8b, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,833 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/4da00d7082974619ad360f2de4544f6e for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,833 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,835 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-d7c3e8037cee4b1f8a229f6d1deb01cd for child: 0380d0a7ec1b73252e4007b78f1d4d8b, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,836 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,836 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/760c76560eee4eefbb87b85f847cdab7 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,836 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-601b759029df437583c54c4428864f3b for child: ed9dbe4102994d49d22b9d46d6f9665c, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,836 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-184d877ecfb14d38b74901982f651776 for child: ed9dbe4102994d49d22b9d46d6f9665c, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,836 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-dd6839ce97cc43d79946989382bcf84e for child: 0380d0a7ec1b73252e4007b78f1d4d8b, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,836 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/dd6839ce97cc43d79946989382bcf84e for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,836 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,836 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-86a0f5fb60a94b639ec16a3cc20bcf8f for child: ed9dbe4102994d49d22b9d46d6f9665c, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,837 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-283a092c4f01419fa02d00e88c1a0973 for child: ed9dbe4102994d49d22b9d46d6f9665c, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,838 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-5c80d8af86af4abe9b56c21af43fe00f for child: 0380d0a7ec1b73252e4007b78f1d4d8b, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,839 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/5c80d8af86af4abe9b56c21af43fe00f for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,839 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,839 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d232db1d45a442e9660bbae9fdf45a1 2023-02-11 19:54:28,840 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-0d4a64a4f45e48da920b3f56a260c645 for child: ed9dbe4102994d49d22b9d46d6f9665c, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,840 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-b03c207e1fba40b0b87ea7e10b9b2bd5 for child: 0380d0a7ec1b73252e4007b78f1d4d8b, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,840 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,844 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-6c7a44bbdaf64a3fbbc220212959a979 for child: 0380d0a7ec1b73252e4007b78f1d4d8b, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,844 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6c7a44bbdaf64a3fbbc220212959a979 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,845 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/184d877ecfb14d38b74901982f651776 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,846 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/283a092c4f01419fa02d00e88c1a0973 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,846 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,846 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/760c76560eee4eefbb87b85f847cdab7, top=false 2023-02-11 19:54:28,847 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/601b759029df437583c54c4428864f3b for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,847 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/0d4a64a4f45e48da920b3f56a260c645 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,877 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b338f1037252481f97f19d11733f2a42 2023-02-11 19:54:28,885 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/.splits/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-760c76560eee4eefbb87b85f847cdab7 for child: ed9dbe4102994d49d22b9d46d6f9665c, parent: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,888 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/760c76560eee4eefbb87b85f847cdab7 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,917 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,918 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,919 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,921 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d232db1d45a442e9660bbae9fdf45a1 2023-02-11 19:54:28,924 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,931 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,936 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,940 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,940 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1 for region: 8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:28,940 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 8e9f8bd96b098a3d9b38f3d7bf6db2fe Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:54:28,979 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-11 19:54:28,981 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-11 19:54:28,982 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676145268982"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676145268982"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676145268982"}]},"ts":"1676145268982"} 2023-02-11 19:54:28,982 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145268982"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145268982"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145268982"}]},"ts":"1676145268982"} 2023-02-11 19:54:28,982 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145268982"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145268982"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145268982"}]},"ts":"1676145268982"} 2023-02-11 19:54:28,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed9dbe4102994d49d22b9d46d6f9665c, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0380d0a7ec1b73252e4007b78f1d4d8b, ASSIGN}] 2023-02-11 19:54:28,986 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0380d0a7ec1b73252e4007b78f1d4d8b, ASSIGN 2023-02-11 19:54:28,987 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0380d0a7ec1b73252e4007b78f1d4d8b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:28,987 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed9dbe4102994d49d22b9d46d6f9665c, ASSIGN 2023-02-11 19:54:28,988 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed9dbe4102994d49d22b9d46d6f9665c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:54:29,137 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 19:54:29,139 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=ed9dbe4102994d49d22b9d46d6f9665c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:29,139 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145269138"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145269138"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145269138"}]},"ts":"1676145269138"} 2023-02-11 19:54:29,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure ed9dbe4102994d49d22b9d46d6f9665c, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:29,142 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=0380d0a7ec1b73252e4007b78f1d4d8b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:29,142 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145269142"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145269142"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145269142"}]},"ts":"1676145269142"} 2023-02-11 19:54:29,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 0380d0a7ec1b73252e4007b78f1d4d8b, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:29,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ed9dbe4102994d49d22b9d46d6f9665c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.', STARTKEY => '', ENDKEY => '60'} 2023-02-11 19:54:29,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:29,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:29,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,297 INFO [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,297 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1 2023-02-11 19:54:29,297 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1 2023-02-11 19:54:29,298 INFO [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed9dbe4102994d49d22b9d46d6f9665c columnFamilyName cf1 2023-02-11 19:54:29,308 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef-bottom 2023-02-11 19:54:29,311 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,312 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1-bottom 2023-02-11 19:54:29,317 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410-bottom 2023-02-11 19:54:29,321 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624-bottom 2023-02-11 19:54:29,324 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-0d4a64a4f45e48da920b3f56a260c645 2023-02-11 19:54:29,327 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-184d877ecfb14d38b74901982f651776 2023-02-11 19:54:29,330 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-760c76560eee4eefbb87b85f847cdab7 2023-02-11 19:54:29,330 INFO [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(310): Store=ed9dbe4102994d49d22b9d46d6f9665c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:29,331 INFO [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,331 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2 2023-02-11 19:54:29,332 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2 2023-02-11 19:54:29,332 INFO [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed9dbe4102994d49d22b9d46d6f9665c columnFamilyName cf2 2023-02-11 19:54:29,343 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7-bottom 2023-02-11 19:54:29,346 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95-bottom 2023-02-11 19:54:29,349 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,350 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42-bottom 2023-02-11 19:54:29,353 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431-bottom 2023-02-11 19:54:29,356 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-283a092c4f01419fa02d00e88c1a0973 2023-02-11 19:54:29,359 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-601b759029df437583c54c4428864f3b 2023-02-11 19:54:29,361 DEBUG [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-86a0f5fb60a94b639ec16a3cc20bcf8f 2023-02-11 19:54:29,362 INFO [StoreOpener-ed9dbe4102994d49d22b9d46d6f9665c-1] regionserver.HStore(310): Store=ed9dbe4102994d49d22b9d46d6f9665c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:29,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,363 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 19:54:29,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:29,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ed9dbe4102994d49d22b9d46d6f9665c; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60624070, jitterRate=-0.09663096070289612}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:29,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ed9dbe4102994d49d22b9d46d6f9665c: Running coprocessor pre-open hook at 1676145269295Writing region info on filesystem at 1676145269295Initializing all the Stores at 1676145269296 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145269296Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145269296Cleaning up temporary data from old regions at 1676145269364 (+68 ms)Cleaning up detritus from prior splits at 1676145269364Running coprocessor post-open hooks at 1676145269367 (+3 ms)Region opened successfully at 1676145269367 2023-02-11 19:54:29,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c., pid=77, masterSystemTime=1676145269292 2023-02-11 19:54:29,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,369 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:29,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,369 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:29,369 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,369 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,370 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): ed9dbe4102994d49d22b9d46d6f9665c/cf2 is initiating minor compaction (all files) 2023-02-11 19:54:29,370 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of ed9dbe4102994d49d22b9d46d6f9665c/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,369 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): ed9dbe4102994d49d22b9d46d6f9665c/cf1 is initiating minor compaction (all files) 2023-02-11 19:54:29,370 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ed9dbe4102994d49d22b9d46d6f9665c/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,370 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-283a092c4f01419fa02d00e88c1a0973, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-86a0f5fb60a94b639ec16a3cc20bcf8f, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-601b759029df437583c54c4428864f3b, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/.tmp, totalSize=35.0 K 2023-02-11 19:54:29,370 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-760c76560eee4eefbb87b85f847cdab7, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-184d877ecfb14d38b74901982f651776, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-0d4a64a4f45e48da920b3f56a260c645, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/.tmp, totalSize=35.0 K 2023-02-11 19:54:29,371 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145266960 2023-02-11 19:54:29,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=ed9dbe4102994d49d22b9d46d6f9665c, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:29,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145269371"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145269371"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145269371"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145269371"}]},"ts":"1676145269371"} 2023-02-11 19:54:29,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0380d0a7ec1b73252e4007b78f1d4d8b, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.', STARTKEY => '60', ENDKEY => ''} 2023-02-11 19:54:29,373 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145266960 2023-02-11 19:54:29,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:29,373 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145267132 2023-02-11 19:54:29,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:29,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,374 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145267132 2023-02-11 19:54:29,375 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145267308 2023-02-11 19:54:29,376 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145267308 2023-02-11 19:54:29,376 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-283a092c4f01419fa02d00e88c1a0973, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145267500 2023-02-11 19:54:29,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-11 19:54:29,377 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-760c76560eee4eefbb87b85f847cdab7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145267500 2023-02-11 19:54:29,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure ed9dbe4102994d49d22b9d46d6f9665c, server=jenkins-hbase4.apache.org,42733,1676145241889 in 234 msec 2023-02-11 19:54:29,378 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-184d877ecfb14d38b74901982f651776, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145267681 2023-02-11 19:54:29,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed9dbe4102994d49d22b9d46d6f9665c, ASSIGN in 392 msec 2023-02-11 19:54:29,378 INFO [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,378 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-86a0f5fb60a94b639ec16a3cc20bcf8f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145267681 2023-02-11 19:54:29,379 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-0d4a64a4f45e48da920b3f56a260c645, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145267856 2023-02-11 19:54:29,379 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-601b759029df437583c54c4428864f3b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145267856 2023-02-11 19:54:29,380 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145268050 2023-02-11 19:54:29,380 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1 2023-02-11 19:54:29,380 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1 2023-02-11 19:54:29,380 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145268050 2023-02-11 19:54:29,389 INFO [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0380d0a7ec1b73252e4007b78f1d4d8b columnFamilyName cf1 2023-02-11 19:54:29,411 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef-top 2023-02-11 19:54:29,412 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): ed9dbe4102994d49d22b9d46d6f9665c#cf2#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:29,417 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,417 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1-top 2023-02-11 19:54:29,421 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): ed9dbe4102994d49d22b9d46d6f9665c#cf1#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:29,421 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410-top 2023-02-11 19:54:29,426 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624-top 2023-02-11 19:54:29,429 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/.tmp/cf2/a6446cbb45b045c2aaffc10c42e52670 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/a6446cbb45b045c2aaffc10c42e52670 2023-02-11 19:54:29,430 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:29,430 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:29,434 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:29,434 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:29,435 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ed9dbe4102994d49d22b9d46d6f9665c/cf2 of ed9dbe4102994d49d22b9d46d6f9665c into a6446cbb45b045c2aaffc10c42e52670(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:29,435 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for ed9dbe4102994d49d22b9d46d6f9665c: 2023-02-11 19:54:29,435 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c., storeName=ed9dbe4102994d49d22b9d46d6f9665c/cf2, priority=9, startTime=1676145269369; duration=0sec 2023-02-11 19:54:29,435 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,438 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:29,438 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:29,438 INFO [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(310): Store=0380d0a7ec1b73252e4007b78f1d4d8b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:29,438 INFO [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,439 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/.tmp/cf1/54552a02aea74308af6b5b9951b3cae9 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/54552a02aea74308af6b5b9951b3cae9 2023-02-11 19:54:29,439 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2 2023-02-11 19:54:29,439 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2 2023-02-11 19:54:29,440 INFO [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0380d0a7ec1b73252e4007b78f1d4d8b columnFamilyName cf2 2023-02-11 19:54:29,444 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ed9dbe4102994d49d22b9d46d6f9665c/cf1 of ed9dbe4102994d49d22b9d46d6f9665c into 54552a02aea74308af6b5b9951b3cae9(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:29,444 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ed9dbe4102994d49d22b9d46d6f9665c: 2023-02-11 19:54:29,444 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c., storeName=ed9dbe4102994d49d22b9d46d6f9665c/cf1, priority=9, startTime=1676145269368; duration=0sec 2023-02-11 19:54:29,444 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,448 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7-top 2023-02-11 19:54:29,451 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95-top 2023-02-11 19:54:29,454 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,454 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42-top 2023-02-11 19:54:29,457 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431-top 2023-02-11 19:54:29,461 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:29,461 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:29,463 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:29,464 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:29,467 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:29,467 DEBUG [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:29,467 INFO [StoreOpener-0380d0a7ec1b73252e4007b78f1d4d8b-1] regionserver.HStore(310): Store=0380d0a7ec1b73252e4007b78f1d4d8b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:29,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:54:29,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0380d0a7ec1b73252e4007b78f1d4d8b; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68053451, jitterRate=0.014075443148612976}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:54:29,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0380d0a7ec1b73252e4007b78f1d4d8b: Running coprocessor pre-open hook at 1676145269373Writing region info on filesystem at 1676145269373Initializing all the Stores at 1676145269378 (+5 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145269378Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145269378Cleaning up temporary data from old regions at 1676145269470 (+92 ms)Cleaning up detritus from prior splits at 1676145269470Running coprocessor post-open hooks at 1676145269472 (+2 ms)Region opened successfully at 1676145269472 2023-02-11 19:54:29,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b., pid=78, masterSystemTime=1676145269292 2023-02-11 19:54:29,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,474 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:29,474 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:54:29,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,474 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,474 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,474 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): 0380d0a7ec1b73252e4007b78f1d4d8b/cf2 is initiating minor compaction (all files) 2023-02-11 19:54:29,474 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): 0380d0a7ec1b73252e4007b78f1d4d8b/cf1 is initiating minor compaction (all files) 2023-02-11 19:54:29,474 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0380d0a7ec1b73252e4007b78f1d4d8b/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,474 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0380d0a7ec1b73252e4007b78f1d4d8b/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,474 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-5c80d8af86af4abe9b56c21af43fe00f, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-4da00d7082974619ad360f2de4544f6e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-6c7a44bbdaf64a3fbbc220212959a979] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/.tmp, totalSize=34.8 K 2023-02-11 19:54:29,474 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-d7c3e8037cee4b1f8a229f6d1deb01cd, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-b03c207e1fba40b0b87ea7e10b9b2bd5, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-dd6839ce97cc43d79946989382bcf84e] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/.tmp, totalSize=34.8 K 2023-02-11 19:54:29,475 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145266960 2023-02-11 19:54:29,475 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145266960 2023-02-11 19:54:29,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,475 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145267132 2023-02-11 19:54:29,475 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145267132 2023-02-11 19:54:29,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,476 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145267308 2023-02-11 19:54:29,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=0380d0a7ec1b73252e4007b78f1d4d8b, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:29,476 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145267308 2023-02-11 19:54:29,476 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145269476"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145269476"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145269476"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145269476"}]},"ts":"1676145269476"} 2023-02-11 19:54:29,476 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676145268050 2023-02-11 19:54:29,476 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676145268050 2023-02-11 19:54:29,477 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-5c80d8af86af4abe9b56c21af43fe00f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-11 19:54:29,477 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-d7c3e8037cee4b1f8a229f6d1deb01cd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-11 19:54:29,477 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-4da00d7082974619ad360f2de4544f6e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-11 19:54:29,477 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-b03c207e1fba40b0b87ea7e10b9b2bd5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-11 19:54:29,477 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-6c7a44bbdaf64a3fbbc220212959a979, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 19:54:29,478 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-dd6839ce97cc43d79946989382bcf84e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 19:54:29,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-11 19:54:29,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 0380d0a7ec1b73252e4007b78f1d4d8b, server=jenkins-hbase4.apache.org,42733,1676145241889 in 333 msec 2023-02-11 19:54:29,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-11 19:54:29,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0380d0a7ec1b73252e4007b78f1d4d8b, ASSIGN in 495 msec 2023-02-11 19:54:29,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b in 913 msec 2023-02-11 19:54:29,493 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): 0380d0a7ec1b73252e4007b78f1d4d8b#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:29,493 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0380d0a7ec1b73252e4007b78f1d4d8b#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:29,512 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 22a07e8c36c6442dbd4c47c3c5a0f7cc: NONE, but ROW specified in column family configuration 2023-02-11 19:54:29,512 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 04f57df273b64bd59d6bec54256153a2: NONE, but ROW specified in column family configuration 2023-02-11 19:54:29,513 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/.tmp/cf1/22a07e8c36c6442dbd4c47c3c5a0f7cc as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/22a07e8c36c6442dbd4c47c3c5a0f7cc 2023-02-11 19:54:29,513 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/.tmp/cf2/04f57df273b64bd59d6bec54256153a2 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/04f57df273b64bd59d6bec54256153a2 2023-02-11 19:54:29,516 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 22a07e8c36c6442dbd4c47c3c5a0f7cc: NONE, but ROW specified in column family configuration 2023-02-11 19:54:29,517 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 04f57df273b64bd59d6bec54256153a2: NONE, but ROW specified in column family configuration 2023-02-11 19:54:29,518 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0380d0a7ec1b73252e4007b78f1d4d8b/cf1 of 0380d0a7ec1b73252e4007b78f1d4d8b into 22a07e8c36c6442dbd4c47c3c5a0f7cc(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:29,518 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0380d0a7ec1b73252e4007b78f1d4d8b/cf2 of 0380d0a7ec1b73252e4007b78f1d4d8b into 04f57df273b64bd59d6bec54256153a2(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:29,518 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0380d0a7ec1b73252e4007b78f1d4d8b: 2023-02-11 19:54:29,518 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0380d0a7ec1b73252e4007b78f1d4d8b: 2023-02-11 19:54:29,518 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b., storeName=0380d0a7ec1b73252e4007b78f1d4d8b/cf2, priority=9, startTime=1676145269474; duration=0sec 2023-02-11 19:54:29,518 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b., storeName=0380d0a7ec1b73252e4007b78f1d4d8b/cf1, priority=9, startTime=1676145269473; duration=0sec 2023-02-11 19:54:29,518 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,518 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,580 DEBUG [Listener at localhost/38771] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 19:54:29,583 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34722, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 19:54:29,583 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,584 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 19:54:29,584 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] regionserver.HStore(1912): ed9dbe4102994d49d22b9d46d6f9665c/cf1 is initiating major compaction (all files) 2023-02-11 19:54:29,585 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@66dfe9e2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,585 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ed9dbe4102994d49d22b9d46d6f9665c/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,585 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 19:54:29,585 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/54552a02aea74308af6b5b9951b3cae9] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/.tmp, totalSize=6.8 K 2023-02-11 19:54:29,585 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] regionserver.HStore(1912): ed9dbe4102994d49d22b9d46d6f9665c/cf2 is initiating major compaction (all files) 2023-02-11 19:54:29,585 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6014b1b1; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,585 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of ed9dbe4102994d49d22b9d46d6f9665c/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,585 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/a6446cbb45b045c2aaffc10c42e52670] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/.tmp, totalSize=6.8 K 2023-02-11 19:54:29,585 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 54552a02aea74308af6b5b9951b3cae9, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145266960 2023-02-11 19:54:29,585 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting a6446cbb45b045c2aaffc10c42e52670, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145266960 2023-02-11 19:54:29,586 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,587 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 19:54:29,587 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] regionserver.HStore(1912): 0380d0a7ec1b73252e4007b78f1d4d8b/cf1 is initiating major compaction (all files) 2023-02-11 19:54:29,587 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@68561fa0; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:54:29,587 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 19:54:29,587 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] regionserver.HStore(1912): 0380d0a7ec1b73252e4007b78f1d4d8b/cf2 is initiating major compaction (all files) 2023-02-11 19:54:29,587 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@698502de; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 19:54:29,588 INFO [Listener at localhost/38771] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-11 19:54:29,590 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): ed9dbe4102994d49d22b9d46d6f9665c#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:29,590 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): ed9dbe4102994d49d22b9d46d6f9665c#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:29,607 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. is not mergeable because it has references 2023-02-11 19:54:29,608 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. is not mergeable because it has references 2023-02-11 19:54:29,611 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/.tmp/cf2/102ce101bbba4632bbf06a6f5f4640b0 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/102ce101bbba4632bbf06a6f5f4640b0 2023-02-11 19:54:29,612 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/.tmp/cf1/545982f704cb41dab991f3aa9b6623fe as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/545982f704cb41dab991f3aa9b6623fe 2023-02-11 19:54:29,616 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in ed9dbe4102994d49d22b9d46d6f9665c/cf2 of ed9dbe4102994d49d22b9d46d6f9665c into 102ce101bbba4632bbf06a6f5f4640b0(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:29,616 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for ed9dbe4102994d49d22b9d46d6f9665c: 2023-02-11 19:54:29,616 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in ed9dbe4102994d49d22b9d46d6f9665c/cf1 of ed9dbe4102994d49d22b9d46d6f9665c into 545982f704cb41dab991f3aa9b6623fe(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:29,616 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ed9dbe4102994d49d22b9d46d6f9665c: 2023-02-11 19:54:29,617 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676145269585; duration=0sec 2023-02-11 19:54:29,617 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676145269584; duration=0sec 2023-02-11 19:54:29,617 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 19:54:29,617 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:54:29,617 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0380d0a7ec1b73252e4007b78f1d4d8b/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,617 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0380d0a7ec1b73252e4007b78f1d4d8b/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,617 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/04f57df273b64bd59d6bec54256153a2] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/.tmp, totalSize=5.0 K 2023-02-11 19:54:29,617 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/22a07e8c36c6442dbd4c47c3c5a0f7cc] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/.tmp, totalSize=5.0 K 2023-02-11 19:54:29,618 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 04f57df273b64bd59d6bec54256153a2, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 19:54:29,618 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 22a07e8c36c6442dbd4c47c3c5a0f7cc, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 19:54:29,620 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0380d0a7ec1b73252e4007b78f1d4d8b#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:29,620 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): 0380d0a7ec1b73252e4007b78f1d4d8b#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:54:29,634 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4ee1ef6dc23948a999dd81b7f164eb59: NONE, but ROW specified in column family configuration 2023-02-11 19:54:29,635 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/.tmp/cf1/4ee1ef6dc23948a999dd81b7f164eb59 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/4ee1ef6dc23948a999dd81b7f164eb59 2023-02-11 19:54:29,636 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 9ba08feed4554b7aa569fd62dc5f8098: NONE, but ROW specified in column family configuration 2023-02-11 19:54:29,636 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/.tmp/cf2/9ba08feed4554b7aa569fd62dc5f8098 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/9ba08feed4554b7aa569fd62dc5f8098 2023-02-11 19:54:29,638 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4ee1ef6dc23948a999dd81b7f164eb59: NONE, but ROW specified in column family configuration 2023-02-11 19:54:29,640 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 0380d0a7ec1b73252e4007b78f1d4d8b/cf1 of 0380d0a7ec1b73252e4007b78f1d4d8b into 4ee1ef6dc23948a999dd81b7f164eb59(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:29,640 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0380d0a7ec1b73252e4007b78f1d4d8b: 2023-02-11 19:54:29,640 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676145269587; duration=0sec 2023-02-11 19:54:29,640 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,640 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 9ba08feed4554b7aa569fd62dc5f8098: NONE, but ROW specified in column family configuration 2023-02-11 19:54:29,641 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 0380d0a7ec1b73252e4007b78f1d4d8b/cf2 of 0380d0a7ec1b73252e4007b78f1d4d8b into 9ba08feed4554b7aa569fd62dc5f8098(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:54:29,641 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0380d0a7ec1b73252e4007b78f1d4d8b: 2023-02-11 19:54:29,641 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676145269587; duration=0sec 2023-02-11 19:54:29,641 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:54:29,715 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42733] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. is not mergeable because it has references 2023-02-11 19:54:29,716 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42733] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. is not mergeable because it has references 2023-02-11 19:54:29,722 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-11 19:54:29,723 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:29,725 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:29,725 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:29,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:29,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 19:54:29,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145269728"}]},"ts":"1676145269728"} 2023-02-11 19:54:29,729 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-11 19:54:29,731 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-11 19:54:29,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed9dbe4102994d49d22b9d46d6f9665c, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0380d0a7ec1b73252e4007b78f1d4d8b, UNASSIGN}] 2023-02-11 19:54:29,732 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0380d0a7ec1b73252e4007b78f1d4d8b, UNASSIGN 2023-02-11 19:54:29,733 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=0380d0a7ec1b73252e4007b78f1d4d8b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:29,733 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145269733"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145269733"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145269733"}]},"ts":"1676145269733"} 2023-02-11 19:54:29,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 0380d0a7ec1b73252e4007b78f1d4d8b, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:29,736 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed9dbe4102994d49d22b9d46d6f9665c, UNASSIGN 2023-02-11 19:54:29,736 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=ed9dbe4102994d49d22b9d46d6f9665c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:29,736 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145269736"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145269736"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145269736"}]},"ts":"1676145269736"} 2023-02-11 19:54:29,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure ed9dbe4102994d49d22b9d46d6f9665c, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:29,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 19:54:29,885 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ed9dbe4102994d49d22b9d46d6f9665c, disabling compactions & flushes 2023-02-11 19:54:29,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. after waiting 0 ms 2023-02-11 19:54:29,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,886 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-760c76560eee4eefbb87b85f847cdab7, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-184d877ecfb14d38b74901982f651776, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-0d4a64a4f45e48da920b3f56a260c645, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/54552a02aea74308af6b5b9951b3cae9, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1-bottom] to archive 2023-02-11 19:54:29,887 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:29,889 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,890 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,891 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,892 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-760c76560eee4eefbb87b85f847cdab7 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-760c76560eee4eefbb87b85f847cdab7 2023-02-11 19:54:29,893 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-184d877ecfb14d38b74901982f651776 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-184d877ecfb14d38b74901982f651776 2023-02-11 19:54:29,894 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-0d4a64a4f45e48da920b3f56a260c645 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-0d4a64a4f45e48da920b3f56a260c645 2023-02-11 19:54:29,895 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/54552a02aea74308af6b5b9951b3cae9 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/54552a02aea74308af6b5b9951b3cae9 2023-02-11 19:54:29,896 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,898 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-283a092c4f01419fa02d00e88c1a0973, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-86a0f5fb60a94b639ec16a3cc20bcf8f, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-601b759029df437583c54c4428864f3b, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/a6446cbb45b045c2aaffc10c42e52670, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42-bottom] to archive 2023-02-11 19:54:29,899 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:29,900 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,901 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,902 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,903 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-283a092c4f01419fa02d00e88c1a0973 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-283a092c4f01419fa02d00e88c1a0973 2023-02-11 19:54:29,904 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-86a0f5fb60a94b639ec16a3cc20bcf8f to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-86a0f5fb60a94b639ec16a3cc20bcf8f 2023-02-11 19:54:29,905 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-601b759029df437583c54c4428864f3b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-601b759029df437583c54c4428864f3b 2023-02-11 19:54:29,906 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/a6446cbb45b045c2aaffc10c42e52670 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/a6446cbb45b045c2aaffc10c42e52670 2023-02-11 19:54:29,907 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-11 19:54:29,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:29,912 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c. 2023-02-11 19:54:29,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ed9dbe4102994d49d22b9d46d6f9665c: Waiting for close lock at 1676145269885Running coprocessor pre-close hooks at 1676145269885Disabling compacts and flushes for region at 1676145269885Disabling writes for close at 1676145269886 (+1 ms)Writing region close event to WAL at 1676145269908 (+22 ms)Running coprocessor post-close hooks at 1676145269911 (+3 ms)Closed at 1676145269912 (+1 ms) 2023-02-11 19:54:29,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:29,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0380d0a7ec1b73252e4007b78f1d4d8b, disabling compactions & flushes 2023-02-11 19:54:29,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. after waiting 0 ms 2023-02-11 19:54:29,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,914 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=ed9dbe4102994d49d22b9d46d6f9665c, regionState=CLOSED 2023-02-11 19:54:29,914 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145269914"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145269914"}]},"ts":"1676145269914"} 2023-02-11 19:54:29,915 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-5c80d8af86af4abe9b56c21af43fe00f, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-4da00d7082974619ad360f2de4544f6e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/22a07e8c36c6442dbd4c47c3c5a0f7cc, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-6c7a44bbdaf64a3fbbc220212959a979] to archive 2023-02-11 19:54:29,915 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:29,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-11 19:54:29,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure ed9dbe4102994d49d22b9d46d6f9665c, server=jenkins-hbase4.apache.org,42733,1676145241889 in 178 msec 2023-02-11 19:54:29,917 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/796b56dfcab34d22bb2864c7f5291410.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed9dbe4102994d49d22b9d46d6f9665c, UNASSIGN in 186 msec 2023-02-11 19:54:29,918 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/215594f0d75d44b8946e075b872883ef.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,919 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/bf508356a55043f281e5a0309d340624.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,920 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/6d232db1d45a442e9660bbae9fdf45a1.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,921 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-5c80d8af86af4abe9b56c21af43fe00f to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:29,922 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-4da00d7082974619ad360f2de4544f6e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:29,922 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/22a07e8c36c6442dbd4c47c3c5a0f7cc to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/22a07e8c36c6442dbd4c47c3c5a0f7cc 2023-02-11 19:54:29,923 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-6c7a44bbdaf64a3fbbc220212959a979 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:29,925 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-d7c3e8037cee4b1f8a229f6d1deb01cd, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-b03c207e1fba40b0b87ea7e10b9b2bd5, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/04f57df273b64bd59d6bec54256153a2, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-dd6839ce97cc43d79946989382bcf84e] to archive 2023-02-11 19:54:29,926 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:54:29,927 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b944ceb15a4248808a1044e45f277431.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,928 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/798f30525b9e45df8a66d6f2bc4eae95.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,929 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,929 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/b338f1037252481f97f19d11733f2a42.8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:29,930 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-d7c3e8037cee4b1f8a229f6d1deb01cd to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:29,931 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-b03c207e1fba40b0b87ea7e10b9b2bd5 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:29,932 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/04f57df273b64bd59d6bec54256153a2 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/04f57df273b64bd59d6bec54256153a2 2023-02-11 19:54:29,933 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-dd6839ce97cc43d79946989382bcf84e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/testSplitTableRegionDeletedRowsDaughter=8e9f8bd96b098a3d9b38f3d7bf6db2fe-dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:29,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-11 19:54:29,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:29,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b. 2023-02-11 19:54:29,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0380d0a7ec1b73252e4007b78f1d4d8b: Waiting for close lock at 1676145269913Running coprocessor pre-close hooks at 1676145269913Disabling compacts and flushes for region at 1676145269913Disabling writes for close at 1676145269913Writing region close event to WAL at 1676145269934 (+21 ms)Running coprocessor post-close hooks at 1676145269937 (+3 ms)Closed at 1676145269938 (+1 ms) 2023-02-11 19:54:29,939 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:29,940 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=0380d0a7ec1b73252e4007b78f1d4d8b, regionState=CLOSED 2023-02-11 19:54:29,940 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676145269940"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145269940"}]},"ts":"1676145269940"} 2023-02-11 19:54:29,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-11 19:54:29,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 0380d0a7ec1b73252e4007b78f1d4d8b, server=jenkins-hbase4.apache.org,42733,1676145241889 in 207 msec 2023-02-11 19:54:29,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-11 19:54:29,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0380d0a7ec1b73252e4007b78f1d4d8b, UNASSIGN in 211 msec 2023-02-11 19:54:29,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145269944"}]},"ts":"1676145269944"} 2023-02-11 19:54:29,945 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-11 19:54:29,947 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-11 19:54:29,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 222 msec 2023-02-11 19:54:30,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 19:54:30,030 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-11 19:54:30,031 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:30,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:30,033 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:30,034 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:30,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-11 19:54:30,038 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:30,038 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:30,038 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:30,040 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/recovered.edits] 2023-02-11 19:54:30,040 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/recovered.edits] 2023-02-11 19:54:30,040 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/recovered.edits] 2023-02-11 19:54:30,042 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/545982f704cb41dab991f3aa9b6623fe to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf1/545982f704cb41dab991f3aa9b6623fe 2023-02-11 19:54:30,042 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/4ee1ef6dc23948a999dd81b7f164eb59 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf1/4ee1ef6dc23948a999dd81b7f164eb59 2023-02-11 19:54:30,042 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/0d4a64a4f45e48da920b3f56a260c645 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/0d4a64a4f45e48da920b3f56a260c645 2023-02-11 19:54:30,044 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/184d877ecfb14d38b74901982f651776 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/184d877ecfb14d38b74901982f651776 2023-02-11 19:54:30,045 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/102ce101bbba4632bbf06a6f5f4640b0 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/cf2/102ce101bbba4632bbf06a6f5f4640b0 2023-02-11 19:54:30,045 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/9ba08feed4554b7aa569fd62dc5f8098 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/cf2/9ba08feed4554b7aa569fd62dc5f8098 2023-02-11 19:54:30,045 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/215594f0d75d44b8946e075b872883ef 2023-02-11 19:54:30,046 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/4da00d7082974619ad360f2de4544f6e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/4da00d7082974619ad360f2de4544f6e 2023-02-11 19:54:30,047 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/recovered.edits/113.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c/recovered.edits/113.seqid 2023-02-11 19:54:30,048 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/recovered.edits/113.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b/recovered.edits/113.seqid 2023-02-11 19:54:30,048 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/5c80d8af86af4abe9b56c21af43fe00f to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/5c80d8af86af4abe9b56c21af43fe00f 2023-02-11 19:54:30,048 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed9dbe4102994d49d22b9d46d6f9665c 2023-02-11 19:54:30,048 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:54:30,049 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6c7a44bbdaf64a3fbbc220212959a979 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6c7a44bbdaf64a3fbbc220212959a979 2023-02-11 19:54:30,050 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/6d232db1d45a442e9660bbae9fdf45a1 2023-02-11 19:54:30,052 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/760c76560eee4eefbb87b85f847cdab7 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/760c76560eee4eefbb87b85f847cdab7 2023-02-11 19:54:30,053 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/796b56dfcab34d22bb2864c7f5291410 2023-02-11 19:54:30,054 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf1/bf508356a55043f281e5a0309d340624 2023-02-11 19:54:30,057 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/0ec71b6d43eb4fb2ad109cd4d9e370d7 2023-02-11 19:54:30,059 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/283a092c4f01419fa02d00e88c1a0973 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/283a092c4f01419fa02d00e88c1a0973 2023-02-11 19:54:30,060 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/601b759029df437583c54c4428864f3b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/601b759029df437583c54c4428864f3b 2023-02-11 19:54:30,062 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/798f30525b9e45df8a66d6f2bc4eae95 2023-02-11 19:54:30,063 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/86a0f5fb60a94b639ec16a3cc20bcf8f 2023-02-11 19:54:30,065 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b03c207e1fba40b0b87ea7e10b9b2bd5 2023-02-11 19:54:30,066 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b338f1037252481f97f19d11733f2a42 2023-02-11 19:54:30,068 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/b944ceb15a4248808a1044e45f277431 2023-02-11 19:54:30,069 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/d7c3e8037cee4b1f8a229f6d1deb01cd 2023-02-11 19:54:30,070 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/dd6839ce97cc43d79946989382bcf84e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/cf2/dd6839ce97cc43d79946989382bcf84e 2023-02-11 19:54:30,073 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/recovered.edits/106.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe/recovered.edits/106.seqid 2023-02-11 19:54:30,074 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8e9f8bd96b098a3d9b38f3d7bf6db2fe 2023-02-11 19:54:30,074 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-11 19:54:30,076 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:30,078 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-11 19:54:30,080 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-11 19:54:30,081 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:30,081 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-11 19:54:30,081 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145270081"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:30,081 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145270081"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:30,081 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145270081"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:30,082 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 19:54:30,083 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8e9f8bd96b098a3d9b38f3d7bf6db2fe, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676145266456.8e9f8bd96b098a3d9b38f3d7bf6db2fe.', STARTKEY => '', ENDKEY => ''}, {ENCODED => ed9dbe4102994d49d22b9d46d6f9665c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676145268569.ed9dbe4102994d49d22b9d46d6f9665c.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 0380d0a7ec1b73252e4007b78f1d4d8b, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676145268569.0380d0a7ec1b73252e4007b78f1d4d8b.', STARTKEY => '60', ENDKEY => ''}] 2023-02-11 19:54:30,083 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-11 19:54:30,083 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145270083"}]},"ts":"9223372036854775807"} 2023-02-11 19:54:30,084 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-11 19:54:30,087 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:54:30,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 56 msec 2023-02-11 19:54:30,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-11 19:54:30,135 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-11 19:54:30,149 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=408 (was 402) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36872 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:53884 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:53864 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:53876 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:44062 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:33894 [Waiting for operation #61] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:53886 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36926 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36908 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36914 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36936 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:44042 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:53966 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36924 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:36874 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-14050709_15 at /127.0.0.1:53872 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xe288499-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=851 (was 844) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=347 (was 377), ProcessCount=168 (was 168), AvailableMemoryMB=3668 (was 3703) 2023-02-11 19:54:30,163 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=408, OpenFileDescriptor=851, MaxFileDescriptor=60000, SystemLoadAverage=347, ProcessCount=168, AvailableMemoryMB=3667 2023-02-11 19:54:30,163 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:54:30,166 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:54:30,166 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:54:30,167 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:54:30,168 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:54:30,169 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:54:30,170 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,171 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 empty. 2023-02-11 19:54:30,171 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,171 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-11 19:54:30,181 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:30,182 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7ced170548c095f8fdf54b9ac1dd6465, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:54:30,188 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:30,188 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 7ced170548c095f8fdf54b9ac1dd6465, disabling compactions & flushes 2023-02-11 19:54:30,189 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,189 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,189 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. after waiting 0 ms 2023-02-11 19:54:30,189 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,189 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,189 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 7ced170548c095f8fdf54b9ac1dd6465: Waiting for close lock at 1676145270188Disabling compacts and flushes for region at 1676145270188Disabling writes for close at 1676145270189 (+1 ms)Writing region close event to WAL at 1676145270189Closed at 1676145270189 2023-02-11 19:54:30,190 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:54:30,191 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145270191"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145270191"}]},"ts":"1676145270191"} 2023-02-11 19:54:30,192 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:54:30,193 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:54:30,193 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145270193"}]},"ts":"1676145270193"} 2023-02-11 19:54:30,194 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-11 19:54:30,202 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:30,203 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:30,203 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:30,203 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:30,203 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:54:30,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, ASSIGN}] 2023-02-11 19:54:30,204 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, ASSIGN 2023-02-11 19:54:30,205 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32837,1676145241806; forceNewPlan=false, retain=false 2023-02-11 19:54:30,355 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:30,357 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=7ced170548c095f8fdf54b9ac1dd6465, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:30,358 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145270357"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145270357"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145270357"}]},"ts":"1676145270357"} 2023-02-11 19:54:30,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 7ced170548c095f8fdf54b9ac1dd6465, server=jenkins-hbase4.apache.org,32837,1676145241806}] 2023-02-11 19:54:30,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7ced170548c095f8fdf54b9ac1dd6465, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:30,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:30,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:30,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,518 INFO [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,520 DEBUG [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/cf1 2023-02-11 19:54:30,520 DEBUG [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/cf1 2023-02-11 19:54:30,520 INFO [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ced170548c095f8fdf54b9ac1dd6465 columnFamilyName cf1 2023-02-11 19:54:30,521 INFO [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] regionserver.HStore(310): Store=7ced170548c095f8fdf54b9ac1dd6465/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:30,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:54:30,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7ced170548c095f8fdf54b9ac1dd6465; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67697633, jitterRate=0.00877334177494049}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 19:54:30,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7ced170548c095f8fdf54b9ac1dd6465: Running coprocessor pre-open hook at 1676145270517Writing region info on filesystem at 1676145270517Initializing all the Stores at 1676145270518 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145270518Cleaning up temporary data from old regions at 1676145270523 (+5 ms)Cleaning up detritus from prior splits at 1676145270523Running coprocessor post-open hooks at 1676145270529 (+6 ms)Region opened successfully at 1676145270529 2023-02-11 19:54:30,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465., pid=87, masterSystemTime=1676145270510 2023-02-11 19:54:30,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,532 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=7ced170548c095f8fdf54b9ac1dd6465, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:30,532 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145270531"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145270531"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145270531"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145270531"}]},"ts":"1676145270531"} 2023-02-11 19:54:30,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-11 19:54:30,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 7ced170548c095f8fdf54b9ac1dd6465, server=jenkins-hbase4.apache.org,32837,1676145241806 in 174 msec 2023-02-11 19:54:30,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-11 19:54:30,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, ASSIGN in 331 msec 2023-02-11 19:54:30,536 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:54:30,536 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145270536"}]},"ts":"1676145270536"} 2023-02-11 19:54:30,537 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-11 19:54:30,542 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:54:30,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 376 msec 2023-02-11 19:54:30,676 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 19:54:30,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:54:30,696 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-11 19:54:30,697 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-11 19:54:30,697 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-11 19:54:30,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-11 19:54:30,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, REOPEN/MOVE}] 2023-02-11 19:54:30,712 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, REOPEN/MOVE 2023-02-11 19:54:30,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7ced170548c095f8fdf54b9ac1dd6465, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:30,713 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145270713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145270713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145270713"}]},"ts":"1676145270713"} 2023-02-11 19:54:30,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 7ced170548c095f8fdf54b9ac1dd6465, server=jenkins-hbase4.apache.org,32837,1676145241806}] 2023-02-11 19:54:30,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7ced170548c095f8fdf54b9ac1dd6465, disabling compactions & flushes 2023-02-11 19:54:30,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. after waiting 0 ms 2023-02-11 19:54:30,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-11 19:54:30,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:54:30,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:30,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7ced170548c095f8fdf54b9ac1dd6465: Waiting for close lock at 1676145270867Running coprocessor pre-close hooks at 1676145270867Disabling compacts and flushes for region at 1676145270867Disabling writes for close at 1676145270868 (+1 ms)Writing region close event to WAL at 1676145270868Running coprocessor post-close hooks at 1676145270871 (+3 ms)Closed at 1676145270871 2023-02-11 19:54:30,871 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 7ced170548c095f8fdf54b9ac1dd6465 to self. 2023-02-11 19:54:30,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:30,873 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7ced170548c095f8fdf54b9ac1dd6465, regionState=CLOSED 2023-02-11 19:54:30,873 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145270873"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145270873"}]},"ts":"1676145270873"} 2023-02-11 19:54:30,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-11 19:54:30,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 7ced170548c095f8fdf54b9ac1dd6465, server=jenkins-hbase4.apache.org,32837,1676145241806 in 160 msec 2023-02-11 19:54:30,876 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,32837,1676145241806; forceNewPlan=false, retain=true 2023-02-11 19:54:31,027 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:54:31,027 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7ced170548c095f8fdf54b9ac1dd6465, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:31,027 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145271027"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145271027"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145271027"}]},"ts":"1676145271027"} 2023-02-11 19:54:31,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 7ced170548c095f8fdf54b9ac1dd6465, server=jenkins-hbase4.apache.org,32837,1676145241806}] 2023-02-11 19:54:31,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:31,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7ced170548c095f8fdf54b9ac1dd6465, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:54:31,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:31,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:31,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:31,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:31,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:31,185 INFO [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:31,185 DEBUG [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/cf1 2023-02-11 19:54:31,185 DEBUG [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/cf1 2023-02-11 19:54:31,186 INFO [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ced170548c095f8fdf54b9ac1dd6465 columnFamilyName cf1 2023-02-11 19:54:31,186 INFO [StoreOpener-7ced170548c095f8fdf54b9ac1dd6465-1] regionserver.HStore(310): Store=7ced170548c095f8fdf54b9ac1dd6465/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:54:31,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:31,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:31,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:54:31,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7ced170548c095f8fdf54b9ac1dd6465; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60921404, jitterRate=-0.09220033884048462}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 19:54:31,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7ced170548c095f8fdf54b9ac1dd6465: Running coprocessor pre-open hook at 1676145271184Writing region info on filesystem at 1676145271184Initializing all the Stores at 1676145271184Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145271184Cleaning up temporary data from old regions at 1676145271188 (+4 ms)Cleaning up detritus from prior splits at 1676145271188Running coprocessor post-open hooks at 1676145271191 (+3 ms)Region opened successfully at 1676145271191 2023-02-11 19:54:31,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465., pid=92, masterSystemTime=1676145271180 2023-02-11 19:54:31,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:31,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:54:31,193 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7ced170548c095f8fdf54b9ac1dd6465, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:54:31,193 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145271193"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145271193"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145271193"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145271193"}]},"ts":"1676145271193"} 2023-02-11 19:54:31,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-11 19:54:31,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 7ced170548c095f8fdf54b9ac1dd6465, server=jenkins-hbase4.apache.org,32837,1676145241806 in 165 msec 2023-02-11 19:54:31,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-11 19:54:31,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, REOPEN/MOVE in 485 msec 2023-02-11 19:54:31,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-11 19:54:31,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 500 msec 2023-02-11 19:54:31,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014, ASSIGN}] 2023-02-11 19:54:31,204 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014, ASSIGN 2023-02-11 19:54:31,204 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 19:54:31,354 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:54:31,355 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:54:31,355 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:54:31,355 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:54:31,355 DEBUG [jenkins-hbase4:37191] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-11 19:54:31,357 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=fcfe336b013ef46034d5376d55802014, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:54:31,357 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145271357"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676145271357"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676145271357"}]},"ts":"1676145271357"} 2023-02-11 19:54:31,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure fcfe336b013ef46034d5376d55802014, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:54:31,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014. 2023-02-11 19:54:31,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fcfe336b013ef46034d5376d55802014, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-11 19:54:31,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:54:31,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica fcfe336b013ef46034d5376d55802014 2023-02-11 19:54:31,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:54:31,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for fcfe336b013ef46034d5376d55802014 2023-02-11 19:54:31,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for fcfe336b013ef46034d5376d55802014 2023-02-11 19:54:31,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:31,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:31,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:31,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:31,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:32,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,217 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-11 19:54:33,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:33,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,267 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 19:54:34,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:34,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:35,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:36,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:37,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:38,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:39,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:40,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:41,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:42,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:43,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:44,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:45,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:46,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:47,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:48,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:49,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:50,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:51,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:52,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:53,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:54,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:55,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:56,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:57,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:58,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:54:59,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:00,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:01,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:02,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,193 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 19:55:03,236 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 19:55:03,244 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 19:55:03,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:03,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:04,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:05,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:06,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:07,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:08,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:09,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:10,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:11,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:12,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:13,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:14,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:15,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:16,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:17,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:18,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:19,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:20,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:21,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:22,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:23,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:24,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:25,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:26,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:27,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:28,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:29,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 19:55:30,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:30,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 19:55:30,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 19:55:31,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 19:55:31,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:31,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 19:55:31,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:32,896 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:32,897 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:32,898 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:32,898 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:32,898 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:32,900 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 7ced170548c095f8fdf54b9ac1dd6465, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:32,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 exec-time=5 msec 2023-02-11 19:55:32,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:33,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:33,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 19:55:33,235 INFO [StoreOpener-fcfe336b013ef46034d5376d55802014-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fcfe336b013ef46034d5376d55802014 2023-02-11 19:55:33,239 DEBUG [StoreOpener-fcfe336b013ef46034d5376d55802014-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/cf1 2023-02-11 19:55:33,239 DEBUG [StoreOpener-fcfe336b013ef46034d5376d55802014-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/cf1 2023-02-11 19:55:33,240 INFO [StoreOpener-fcfe336b013ef46034d5376d55802014-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fcfe336b013ef46034d5376d55802014 columnFamilyName cf1 2023-02-11 19:55:33,241 INFO [StoreOpener-fcfe336b013ef46034d5376d55802014-1] regionserver.HStore(310): Store=fcfe336b013ef46034d5376d55802014/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:33,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened fcfe336b013ef46034d5376d55802014; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74487363, jitterRate=0.10994820296764374}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 19:55:33,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for fcfe336b013ef46034d5376d55802014: Running coprocessor pre-open hook at 1676145271515Initializing all the Stores at 1676145333234 (+61719 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145333234Running coprocessor post-open hooks at 1676145333242 (+8 ms)Region opened successfully at 1676145333242 2023-02-11 19:55:33,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014., pid=94, masterSystemTime=1676145271512 2023-02-11 19:55:33,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014. 2023-02-11 19:55:33,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014. 2023-02-11 19:55:33,244 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=fcfe336b013ef46034d5376d55802014, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:33,244 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145333244"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676145333244"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676145333244"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676145333244"}]},"ts":"1676145333244"} 2023-02-11 19:55:33,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-11 19:55:33,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure fcfe336b013ef46034d5376d55802014, server=jenkins-hbase4.apache.org,42733,1676145241889 in 1 mins, 1.887 sec 2023-02-11 19:55:33,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-11 19:55:33,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014, ASSIGN in 1 mins, 2.045 sec 2023-02-11 19:55:33,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.571 sec 2023-02-11 19:55:33,251 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676145270673","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62578,"client":"172.31.14.131:41904","queuetimems":0,"class":"HMaster"} 2023-02-11 19:55:33,252 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-11 19:55:35,148 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up 2023-02-11 19:55:35,150 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:35,151 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,151 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 19:55:35,154 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145335154"}]},"ts":"1676145335154"} 2023-02-11 19:55:35,155 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-11 19:55:35,157 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-11 19:55:35,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014, UNASSIGN}] 2023-02-11 19:55:35,158 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014, UNASSIGN 2023-02-11 19:55:35,159 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=fcfe336b013ef46034d5376d55802014, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:35,159 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145335159"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676145335159"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676145335159"}]},"ts":"1676145335159"} 2023-02-11 19:55:35,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure fcfe336b013ef46034d5376d55802014, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:35,161 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, UNASSIGN 2023-02-11 19:55:35,162 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=7ced170548c095f8fdf54b9ac1dd6465, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:35,162 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145335162"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145335162"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145335162"}]},"ts":"1676145335162"} 2023-02-11 19:55:35,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 7ced170548c095f8fdf54b9ac1dd6465, server=jenkins-hbase4.apache.org,32837,1676145241806}] 2023-02-11 19:55:35,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 19:55:35,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close fcfe336b013ef46034d5376d55802014 2023-02-11 19:55:35,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing fcfe336b013ef46034d5376d55802014, disabling compactions & flushes 2023-02-11 19:55:35,312 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014. 2023-02-11 19:55:35,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014. 2023-02-11 19:55:35,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014. after waiting 0 ms 2023-02-11 19:55:35,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014. 2023-02-11 19:55:35,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:35,314 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014. 2023-02-11 19:55:35,314 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for fcfe336b013ef46034d5376d55802014: Waiting for close lock at 1676145335312Running coprocessor pre-close hooks at 1676145335312Disabling compacts and flushes for region at 1676145335312Disabling writes for close at 1676145335313 (+1 ms)Writing region close event to WAL at 1676145335313Running coprocessor post-close hooks at 1676145335313Closed at 1676145335314 (+1 ms) 2023-02-11 19:55:35,315 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:55:35,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7ced170548c095f8fdf54b9ac1dd6465, disabling compactions & flushes 2023-02-11 19:55:35,315 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:55:35,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:55:35,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. after waiting 0 ms 2023-02-11 19:55:35,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:55:35,316 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed fcfe336b013ef46034d5376d55802014 2023-02-11 19:55:35,316 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=fcfe336b013ef46034d5376d55802014, regionState=CLOSED 2023-02-11 19:55:35,316 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145335316"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676145335316"}]},"ts":"1676145335316"} 2023-02-11 19:55:35,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 19:55:35,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:35,321 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465. 2023-02-11 19:55:35,321 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7ced170548c095f8fdf54b9ac1dd6465: Waiting for close lock at 1676145335315Running coprocessor pre-close hooks at 1676145335315Disabling compacts and flushes for region at 1676145335315Disabling writes for close at 1676145335315Writing region close event to WAL at 1676145335316 (+1 ms)Running coprocessor post-close hooks at 1676145335320 (+4 ms)Closed at 1676145335321 (+1 ms) 2023-02-11 19:55:35,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-11 19:55:35,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure fcfe336b013ef46034d5376d55802014, server=jenkins-hbase4.apache.org,42733,1676145241889 in 159 msec 2023-02-11 19:55:35,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:55:35,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014, UNASSIGN in 164 msec 2023-02-11 19:55:35,323 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=7ced170548c095f8fdf54b9ac1dd6465, regionState=CLOSED 2023-02-11 19:55:35,323 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676145335323"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145335323"}]},"ts":"1676145335323"} 2023-02-11 19:55:35,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-11 19:55:35,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 7ced170548c095f8fdf54b9ac1dd6465, server=jenkins-hbase4.apache.org,32837,1676145241806 in 161 msec 2023-02-11 19:55:35,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-11 19:55:35,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7ced170548c095f8fdf54b9ac1dd6465, UNASSIGN in 168 msec 2023-02-11 19:55:35,328 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145335327"}]},"ts":"1676145335327"} 2023-02-11 19:55:35,328 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-11 19:55:35,331 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-11 19:55:35,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 180 msec 2023-02-11 19:55:35,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 19:55:35,456 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-11 19:55:35,457 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,460 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-11 19:55:35,464 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:55:35,466 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/recovered.edits] 2023-02-11 19:55:35,470 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/recovered.edits/7.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465/recovered.edits/7.seqid 2023-02-11 19:55:35,470 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7ced170548c095f8fdf54b9ac1dd6465 2023-02-11 19:55:35,470 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-11 19:55:35,472 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,474 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-11 19:55:35,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-11 19:55:35,477 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,477 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-11 19:55:35,477 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145335477"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:35,477 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145335477"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:35,479 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-11 19:55:35,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7ced170548c095f8fdf54b9ac1dd6465, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676145270166.7ced170548c095f8fdf54b9ac1dd6465.', STARTKEY => '', ENDKEY => ''}, {ENCODED => fcfe336b013ef46034d5376d55802014, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676145270166_0001.fcfe336b013ef46034d5376d55802014.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-11 19:55:35,479 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-11 19:55:35,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145335479"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:35,481 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-11 19:55:35,482 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:35,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 25 msec 2023-02-11 19:55:35,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-11 19:55:35,562 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-11 19:55:35,574 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=347 (was 408), OpenFileDescriptor=658 (was 851), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=151 (was 347), ProcessCount=168 (was 168), AvailableMemoryMB=3454 (was 3667) 2023-02-11 19:55:35,587 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=348, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=151, ProcessCount=168, AvailableMemoryMB=3453 2023-02-11 19:55:35,587 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:35,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:55:35,590 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:55:35,591 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:35,592 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:55:35,593 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:55:35,594 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,595 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a empty. 2023-02-11 19:55:35,595 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,595 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-11 19:55:35,604 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-11 19:55:35,605 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => ab0757506a3db9d16451ff4fe5ca807a, NAME => 'testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:55:35,611 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:35,612 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing ab0757506a3db9d16451ff4fe5ca807a, disabling compactions & flushes 2023-02-11 19:55:35,612 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:35,612 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:35,612 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. after waiting 0 ms 2023-02-11 19:55:35,612 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:35,612 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:35,612 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for ab0757506a3db9d16451ff4fe5ca807a: Waiting for close lock at 1676145335612Disabling compacts and flushes for region at 1676145335612Disabling writes for close at 1676145335612Writing region close event to WAL at 1676145335612Closed at 1676145335612 2023-02-11 19:55:35,613 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:55:35,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676145335614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145335614"}]},"ts":"1676145335614"} 2023-02-11 19:55:35,615 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:55:35,616 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:55:35,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145335616"}]},"ts":"1676145335616"} 2023-02-11 19:55:35,617 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-11 19:55:35,620 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:55:35,621 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:55:35,621 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:55:35,621 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:55:35,621 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:55:35,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ab0757506a3db9d16451ff4fe5ca807a, ASSIGN}] 2023-02-11 19:55:35,622 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ab0757506a3db9d16451ff4fe5ca807a, ASSIGN 2023-02-11 19:55:35,622 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=ab0757506a3db9d16451ff4fe5ca807a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32837,1676145241806; forceNewPlan=false, retain=false 2023-02-11 19:55:35,773 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:55:35,774 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=ab0757506a3db9d16451ff4fe5ca807a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:35,774 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676145335774"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145335774"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145335774"}]},"ts":"1676145335774"} 2023-02-11 19:55:35,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure ab0757506a3db9d16451ff4fe5ca807a, server=jenkins-hbase4.apache.org,32837,1676145241806}] 2023-02-11 19:55:35,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:35,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ab0757506a3db9d16451ff4fe5ca807a, NAME => 'testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:55:35,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:35,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:35,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,931 INFO [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,933 DEBUG [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1 2023-02-11 19:55:35,933 DEBUG [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1 2023-02-11 19:55:35,933 INFO [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab0757506a3db9d16451ff4fe5ca807a columnFamilyName cf1 2023-02-11 19:55:35,934 INFO [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] regionserver.HStore(310): Store=ab0757506a3db9d16451ff4fe5ca807a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:35,934 INFO [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,935 DEBUG [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2 2023-02-11 19:55:35,935 DEBUG [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2 2023-02-11 19:55:35,936 INFO [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab0757506a3db9d16451ff4fe5ca807a columnFamilyName cf2 2023-02-11 19:55:35,936 INFO [StoreOpener-ab0757506a3db9d16451ff4fe5ca807a-1] regionserver.HStore(310): Store=ab0757506a3db9d16451ff4fe5ca807a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:35,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:35,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:35,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:55:35,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ab0757506a3db9d16451ff4fe5ca807a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72359519, jitterRate=0.07824085652828217}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:35,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ab0757506a3db9d16451ff4fe5ca807a: Running coprocessor pre-open hook at 1676145335930Writing region info on filesystem at 1676145335930Initializing all the Stores at 1676145335931 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145335931Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145335931Cleaning up temporary data from old regions at 1676145335938 (+7 ms)Cleaning up detritus from prior splits at 1676145335939 (+1 ms)Running coprocessor post-open hooks at 1676145335943 (+4 ms)Region opened successfully at 1676145335943 2023-02-11 19:55:35,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a., pid=104, masterSystemTime=1676145335927 2023-02-11 19:55:35,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:35,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:35,945 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=ab0757506a3db9d16451ff4fe5ca807a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:35,945 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676145335945"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145335945"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145335945"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145335945"}]},"ts":"1676145335945"} 2023-02-11 19:55:35,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-11 19:55:35,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure ab0757506a3db9d16451ff4fe5ca807a, server=jenkins-hbase4.apache.org,32837,1676145241806 in 172 msec 2023-02-11 19:55:35,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-11 19:55:35,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=ab0757506a3db9d16451ff4fe5ca807a, ASSIGN in 327 msec 2023-02-11 19:55:35,951 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:55:35,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145335951"}]},"ts":"1676145335951"} 2023-02-11 19:55:35,952 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-11 19:55:35,953 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:55:35,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 363 msec 2023-02-11 19:55:36,098 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:36,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 19:55:36,100 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 19:55:36,100 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,100 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 19:55:36,100 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 19:55:36,101 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,101 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,102 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,102 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,102 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,102 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,102 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,102 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,103 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,103 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,104 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,104 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,104 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. started... 2023-02-11 19:55:36,105 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab0757506a3db9d16451ff4fe5ca807a 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 19:55:36,106 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,114 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/76533582d9444bf6963d3ef087e06953 2023-02-11 19:55:36,126 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/04a2363dac504441b3582784057d8853 2023-02-11 19:55:36,129 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/76533582d9444bf6963d3ef087e06953 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/76533582d9444bf6963d3ef087e06953 2023-02-11 19:55:36,133 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/76533582d9444bf6963d3ef087e06953, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:36,133 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/04a2363dac504441b3582784057d8853 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/04a2363dac504441b3582784057d8853 2023-02-11 19:55:36,137 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/04a2363dac504441b3582784057d8853, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:36,137 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ab0757506a3db9d16451ff4fe5ca807a in 32ms, sequenceid=6, compaction requested=false 2023-02-11 19:55:36,137 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-11 19:55:36,138 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab0757506a3db9d16451ff4fe5ca807a: 2023-02-11 19:55:36,138 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,138 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:36,138 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:36,138 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,138 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,138 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,139 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,139 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,140 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,140 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 19:55:36,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28f8856a[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:36,142 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 19:55:36,142 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,143 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,143 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:36,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,143 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:36,145 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,145 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,145 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,145 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,32837,1676145241806': 2023-02-11 19:55:36,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 19:55:36,149 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 19:55:36,149 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:36,149 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:36,149 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 19:55:36,149 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:36,151 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,151 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,151 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,151 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,152 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,152 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,154 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,154 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,154 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,157 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,157 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 19:55:36,157 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:36,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:36,157 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,158 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,158 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,158 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,158 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,158 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,158 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,258 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:36,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:36,269 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:36,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:36,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 19:55:36,271 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 19:55:36,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 19:55:36,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 19:55:36,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,272 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,273 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,273 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,272 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,273 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,273 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:36,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,274 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,274 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,274 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,274 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,275 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:36,275 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:36,275 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,275 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. started... 2023-02-11 19:55:36,275 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab0757506a3db9d16451ff4fe5ca807a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:36,277 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,281 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,281 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,281 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,284 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/bcec54863d954a6292d44343b5e9fad3 2023-02-11 19:55:36,295 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/7a3aab1b2fa044cd98d310f9c284da68 2023-02-11 19:55:36,299 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/bcec54863d954a6292d44343b5e9fad3 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/bcec54863d954a6292d44343b5e9fad3 2023-02-11 19:55:36,302 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/bcec54863d954a6292d44343b5e9fad3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:55:36,303 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/7a3aab1b2fa044cd98d310f9c284da68 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/7a3aab1b2fa044cd98d310f9c284da68 2023-02-11 19:55:36,307 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/7a3aab1b2fa044cd98d310f9c284da68, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:55:36,308 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab0757506a3db9d16451ff4fe5ca807a in 33ms, sequenceid=19, compaction requested=false 2023-02-11 19:55:36,308 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab0757506a3db9d16451ff4fe5ca807a: 2023-02-11 19:55:36,308 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,308 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:36,308 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:36,308 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,308 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,308 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,310 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,310 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,310 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,310 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 19:55:36,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c17ece8[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:36,312 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 19:55:36,312 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,314 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,314 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,314 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,315 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,315 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:36,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:36,315 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:36,316 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,316 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,316 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,316 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,316 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,316 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,317 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,317 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,317 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,317 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,32837,1676145241806': 2023-02-11 19:55:36,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 19:55:36,320 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 19:55:36,320 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:36,320 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:36,320 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 19:55:36,320 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:36,322 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,322 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,322 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,322 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,322 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,322 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,322 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,325 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,325 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,326 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,331 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 19:55:36,331 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:36,331 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,332 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:36,332 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:36,332 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,332 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,432 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:36,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:36,444 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:36,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:36,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 19:55:36,445 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 19:55:36,445 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,445 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 19:55:36,445 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 19:55:36,446 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,446 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,447 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,447 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,447 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,447 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,447 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,447 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:36,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,449 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,449 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,449 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,449 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,449 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:36,449 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,449 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,449 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. started... 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,449 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab0757506a3db9d16451ff4fe5ca807a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:36,450 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,451 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,452 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,452 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,452 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,452 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,458 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/17f957384ca34c1f865cfbd8786bfb93 2023-02-11 19:55:36,469 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/564143a7f4354d4fab2337665c7bfd74 2023-02-11 19:55:36,472 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/17f957384ca34c1f865cfbd8786bfb93 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/17f957384ca34c1f865cfbd8786bfb93 2023-02-11 19:55:36,476 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/17f957384ca34c1f865cfbd8786bfb93, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:55:36,477 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/564143a7f4354d4fab2337665c7bfd74 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/564143a7f4354d4fab2337665c7bfd74 2023-02-11 19:55:36,480 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/564143a7f4354d4fab2337665c7bfd74, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:55:36,481 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab0757506a3db9d16451ff4fe5ca807a in 32ms, sequenceid=32, compaction requested=true 2023-02-11 19:55:36,481 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab0757506a3db9d16451ff4fe5ca807a: 2023-02-11 19:55:36,481 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,483 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 19:55:36,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@191c4577[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:36,485 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 19:55:36,485 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,486 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,486 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,487 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,487 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,487 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,487 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,487 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,487 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:36,487 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:36,488 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,488 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,488 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,489 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,489 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,489 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,489 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,489 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,489 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,489 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,32837,1676145241806': 2023-02-11 19:55:36,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 19:55:36,492 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 19:55:36,492 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:36,492 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:36,492 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 19:55:36,492 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:36,495 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,495 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,495 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,495 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,496 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,496 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,498 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,498 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,498 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,503 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,503 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,503 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,503 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,503 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 19:55:36,504 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:36,503 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,503 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,503 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,503 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,504 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,504 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:36,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,504 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,504 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,504 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,504 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,504 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,504 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,505 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,505 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,505 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,505 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,604 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:36,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:36,616 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:36,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:36,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 19:55:36,617 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 19:55:36,618 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,618 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 19:55:36,618 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 19:55:36,618 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,618 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,620 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,620 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,620 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,620 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,620 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,620 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:36,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:36,622 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:36,622 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,622 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. started... 2023-02-11 19:55:36,622 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab0757506a3db9d16451ff4fe5ca807a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:36,624 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,624 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,625 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,625 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,625 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,625 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,630 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/4764b4dc8de2476a8754783eda56bde8 2023-02-11 19:55:36,640 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/0c0fef9dc54f4202a5f697af62669b07 2023-02-11 19:55:36,643 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/4764b4dc8de2476a8754783eda56bde8 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/4764b4dc8de2476a8754783eda56bde8 2023-02-11 19:55:36,647 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/4764b4dc8de2476a8754783eda56bde8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:55:36,647 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/0c0fef9dc54f4202a5f697af62669b07 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/0c0fef9dc54f4202a5f697af62669b07 2023-02-11 19:55:36,651 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/0c0fef9dc54f4202a5f697af62669b07, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:55:36,651 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab0757506a3db9d16451ff4fe5ca807a in 29ms, sequenceid=45, compaction requested=true 2023-02-11 19:55:36,651 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab0757506a3db9d16451ff4fe5ca807a: 2023-02-11 19:55:36,651 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,651 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:36,652 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:36,652 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,652 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,652 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,653 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,653 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,653 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,653 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 19:55:36,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1429ba63[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:36,655 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 19:55:36,655 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,656 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,656 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,656 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:36,656 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,656 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:36,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,656 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,656 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:36,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,657 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,657 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,657 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:36,658 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,658 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,658 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,660 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,660 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,660 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,660 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,660 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,660 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,660 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,32837,1676145241806': 2023-02-11 19:55:36,663 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 19:55:36,663 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 19:55:36,663 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:36,663 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:36,663 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 19:55:36,663 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:36,664 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,664 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,664 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,665 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,665 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,667 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,668 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,668 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,669 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,671 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 19:55:36,671 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,671 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,672 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:36,672 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:36,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,671 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,772 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:36,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:36,783 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:36,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:36,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 19:55:36,784 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 19:55:36,784 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,784 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 19:55:36,784 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 19:55:36,785 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,785 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,786 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,786 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,786 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,786 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,786 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,786 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:36,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,787 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,788 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,788 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,788 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,788 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,788 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,788 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:36,788 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,788 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. started... 2023-02-11 19:55:36,788 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab0757506a3db9d16451ff4fe5ca807a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:36,790 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,797 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/8ae3d11e8620450399cbf0a3f3264562 2023-02-11 19:55:36,806 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/23c9569cf8a54176924f313f343bd30b 2023-02-11 19:55:36,810 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/8ae3d11e8620450399cbf0a3f3264562 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/8ae3d11e8620450399cbf0a3f3264562 2023-02-11 19:55:36,813 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/8ae3d11e8620450399cbf0a3f3264562, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:55:36,814 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/23c9569cf8a54176924f313f343bd30b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/23c9569cf8a54176924f313f343bd30b 2023-02-11 19:55:36,817 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/23c9569cf8a54176924f313f343bd30b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:55:36,817 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab0757506a3db9d16451ff4fe5ca807a in 29ms, sequenceid=58, compaction requested=true 2023-02-11 19:55:36,817 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab0757506a3db9d16451ff4fe5ca807a: 2023-02-11 19:55:36,817 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,817 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:36,817 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:36,817 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,818 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,818 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,819 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,819 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,819 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,819 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 19:55:36,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7816be59[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:36,821 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 19:55:36,821 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,823 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,823 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:36,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,823 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:36,828 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,828 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,828 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,828 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,828 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,828 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,831 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:36,831 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,831 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,832 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:36,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,32837,1676145241806': 2023-02-11 19:55:36,835 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 19:55:36,835 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 19:55:36,835 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:36,835 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:36,835 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 19:55:36,835 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:36,837 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,837 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,838 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,838 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,840 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,840 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,841 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,844 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,844 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 19:55:36,845 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:36,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,844 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:36,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:36,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,845 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:36,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:36,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:36,945 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:36,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:36,956 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:36,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:36,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 19:55:36,958 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 19:55:36,958 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,958 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 19:55:36,958 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 19:55:36,958 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,958 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,960 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,960 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,960 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,960 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:36,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:36,960 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,960 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 19:55:36,961 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,961 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,961 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 19:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 19:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 19:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:36,962 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:36,962 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,962 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. started... 2023-02-11 19:55:36,962 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab0757506a3db9d16451ff4fe5ca807a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:36,963 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,963 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,963 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,963 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,963 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,963 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,971 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/9d032465d2254875b50454aa8be06804 2023-02-11 19:55:36,981 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/766d855622574ec5858813a3235a139c 2023-02-11 19:55:36,985 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/9d032465d2254875b50454aa8be06804 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/9d032465d2254875b50454aa8be06804 2023-02-11 19:55:36,988 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/9d032465d2254875b50454aa8be06804, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:55:36,989 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/766d855622574ec5858813a3235a139c as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/766d855622574ec5858813a3235a139c 2023-02-11 19:55:36,992 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/766d855622574ec5858813a3235a139c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:55:36,993 INFO [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab0757506a3db9d16451ff4fe5ca807a in 31ms, sequenceid=71, compaction requested=true 2023-02-11 19:55:36,993 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab0757506a3db9d16451ff4fe5ca807a: 2023-02-11 19:55:36,993 DEBUG [rs(jenkins-hbase4.apache.org,32837,1676145241806)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:36,993 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:36,993 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:36,993 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,993 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 19:55:36,993 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 19:55:36,994 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,995 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:36,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:36,995 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,995 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:36,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:36,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:36,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:36,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:36,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:36,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:36,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:36,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 19:55:36,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b957d1f[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:36,997 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 19:55:36,997 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:37,000 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,000 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:37,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:36,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:36,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:37,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:37,000 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:37,001 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:37,001 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:37,001 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:37,002 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:37,002 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 19:55:37,002 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:37,002 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:37,002 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,002 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:37,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:37,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:37,002 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 19:55:37,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:37,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:37,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:37,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:37,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:37,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:37,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:37,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,32837,1676145241806': 2023-02-11 19:55:37,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 19:55:37,005 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 19:55:37,005 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:37,005 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:37,005 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 19:55:37,005 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:37,006 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,006 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,006 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,006 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:37,006 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,006 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:37,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:37,007 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:37,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,007 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:37,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:37,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:37,007 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:37,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:37,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:37,008 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:37,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:37,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:37,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,009 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:37,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:37,009 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 19:55:37,010 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:37,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:37,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:37,013 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 19:55:37,013 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:37,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:37,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:37,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:37,013 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:37,014 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 19:55:37,014 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:37,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:37,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:37,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:37,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:37,014 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:37,014 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:37,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:37,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:37,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:37,014 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:37,014 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:37,014 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 19:55:37,014 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 19:55:37,114 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:37,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:37,122 INFO [Listener at localhost/38771] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,123 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.StoreUtils(129): cannot split hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/bcec54863d954a6292d44343b5e9fad3 because midkey is the same as first or last row 2023-02-11 19:55:37,123 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.StoreUtils(129): cannot split hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/7a3aab1b2fa044cd98d310f9c284da68 because midkey is the same as first or last row 2023-02-11 19:55:37,123 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.HRegion(2744): Flushing ab0757506a3db9d16451ff4fe5ca807a 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 19:55:37,132 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/26ae484fa82949bfacaac5b7fe6449d7 2023-02-11 19:55:37,147 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/a9c87b3af56b415fa641d1562f41ec85 2023-02-11 19:55:37,150 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf1/26ae484fa82949bfacaac5b7fe6449d7 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/26ae484fa82949bfacaac5b7fe6449d7 2023-02-11 19:55:37,154 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/26ae484fa82949bfacaac5b7fe6449d7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:55:37,154 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/.tmp/cf2/a9c87b3af56b415fa641d1562f41ec85 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/a9c87b3af56b415fa641d1562f41ec85 2023-02-11 19:55:37,158 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/a9c87b3af56b415fa641d1562f41ec85, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:55:37,158 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ab0757506a3db9d16451ff4fe5ca807a in 35ms, sequenceid=82, compaction requested=true 2023-02-11 19:55:37,159 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.HRegion(2445): Flush status journal for ab0757506a3db9d16451ff4fe5ca807a: 2023-02-11 19:55:37,159 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.StoreUtils(129): cannot split hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/bcec54863d954a6292d44343b5e9fad3 because midkey is the same as first or last row 2023-02-11 19:55:37,159 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=32837] regionserver.StoreUtils(129): cannot split hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/7a3aab1b2fa044cd98d310f9c284da68 because midkey is the same as first or last row 2023-02-11 19:55:37,160 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,160 DEBUG [Listener at localhost/38771] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-11 19:55:37,160 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:37,162 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-11 19:55:37,162 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-11 19:55:37,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:37,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 19:55:37,165 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145337165"}]},"ts":"1676145337165"} 2023-02-11 19:55:37,166 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-11 19:55:37,168 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-11 19:55:37,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ab0757506a3db9d16451ff4fe5ca807a, UNASSIGN}] 2023-02-11 19:55:37,169 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ab0757506a3db9d16451ff4fe5ca807a, UNASSIGN 2023-02-11 19:55:37,169 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=ab0757506a3db9d16451ff4fe5ca807a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:37,170 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676145337169"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145337169"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145337169"}]},"ts":"1676145337169"} 2023-02-11 19:55:37,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure ab0757506a3db9d16451ff4fe5ca807a, server=jenkins-hbase4.apache.org,32837,1676145241806}] 2023-02-11 19:55:37,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 19:55:37,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:37,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ab0757506a3db9d16451ff4fe5ca807a, disabling compactions & flushes 2023-02-11 19:55:37,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:37,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:37,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. after waiting 0 ms 2023-02-11 19:55:37,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:37,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 19:55:37,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:37,326 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a. 2023-02-11 19:55:37,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ab0757506a3db9d16451ff4fe5ca807a: Waiting for close lock at 1676145337322Running coprocessor pre-close hooks at 1676145337322Disabling compacts and flushes for region at 1676145337322Disabling writes for close at 1676145337322Writing region close event to WAL at 1676145337323 (+1 ms)Running coprocessor post-close hooks at 1676145337326 (+3 ms)Closed at 1676145337326 2023-02-11 19:55:37,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:37,328 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=ab0757506a3db9d16451ff4fe5ca807a, regionState=CLOSED 2023-02-11 19:55:37,328 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676145337328"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145337328"}]},"ts":"1676145337328"} 2023-02-11 19:55:37,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-11 19:55:37,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure ab0757506a3db9d16451ff4fe5ca807a, server=jenkins-hbase4.apache.org,32837,1676145241806 in 159 msec 2023-02-11 19:55:37,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-11 19:55:37,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=ab0757506a3db9d16451ff4fe5ca807a, UNASSIGN in 162 msec 2023-02-11 19:55:37,332 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145337332"}]},"ts":"1676145337332"} 2023-02-11 19:55:37,333 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-11 19:55:37,334 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-11 19:55:37,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 173 msec 2023-02-11 19:55:37,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 19:55:37,467 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-11 19:55:37,468 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-11 19:55:37,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:37,470 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:37,471 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:37,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-11 19:55:37,474 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:37,475 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/recovered.edits] 2023-02-11 19:55:37,478 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/17f957384ca34c1f865cfbd8786bfb93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/17f957384ca34c1f865cfbd8786bfb93 2023-02-11 19:55:37,479 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/26ae484fa82949bfacaac5b7fe6449d7 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/26ae484fa82949bfacaac5b7fe6449d7 2023-02-11 19:55:37,480 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/4764b4dc8de2476a8754783eda56bde8 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/4764b4dc8de2476a8754783eda56bde8 2023-02-11 19:55:37,482 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/76533582d9444bf6963d3ef087e06953 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/76533582d9444bf6963d3ef087e06953 2023-02-11 19:55:37,483 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/8ae3d11e8620450399cbf0a3f3264562 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/8ae3d11e8620450399cbf0a3f3264562 2023-02-11 19:55:37,484 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/9d032465d2254875b50454aa8be06804 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/9d032465d2254875b50454aa8be06804 2023-02-11 19:55:37,485 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/bcec54863d954a6292d44343b5e9fad3 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf1/bcec54863d954a6292d44343b5e9fad3 2023-02-11 19:55:37,487 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/04a2363dac504441b3582784057d8853 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/04a2363dac504441b3582784057d8853 2023-02-11 19:55:37,488 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/0c0fef9dc54f4202a5f697af62669b07 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/0c0fef9dc54f4202a5f697af62669b07 2023-02-11 19:55:37,489 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/23c9569cf8a54176924f313f343bd30b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/23c9569cf8a54176924f313f343bd30b 2023-02-11 19:55:37,489 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/564143a7f4354d4fab2337665c7bfd74 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/564143a7f4354d4fab2337665c7bfd74 2023-02-11 19:55:37,490 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/766d855622574ec5858813a3235a139c to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/766d855622574ec5858813a3235a139c 2023-02-11 19:55:37,491 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/7a3aab1b2fa044cd98d310f9c284da68 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/7a3aab1b2fa044cd98d310f9c284da68 2023-02-11 19:55:37,492 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/a9c87b3af56b415fa641d1562f41ec85 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/cf2/a9c87b3af56b415fa641d1562f41ec85 2023-02-11 19:55:37,495 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/recovered.edits/85.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a/recovered.edits/85.seqid 2023-02-11 19:55:37,495 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testInvalidSplitKey/ab0757506a3db9d16451ff4fe5ca807a 2023-02-11 19:55:37,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-11 19:55:37,497 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:37,499 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-11 19:55:37,502 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-11 19:55:37,503 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:37,503 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-11 19:55:37,503 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145337503"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:37,504 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 19:55:37,504 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ab0757506a3db9d16451ff4fe5ca807a, NAME => 'testInvalidSplitKey,,1676145335590.ab0757506a3db9d16451ff4fe5ca807a.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 19:55:37,504 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-11 19:55:37,505 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145337504"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:37,506 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-11 19:55:37,512 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:37,513 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 44 msec 2023-02-11 19:55:37,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-11 19:55:37,572 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-11 19:55:37,585 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=368 (was 348) - Thread LEAK? -, OpenFileDescriptor=747 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=138 (was 151), ProcessCount=168 (was 168), AvailableMemoryMB=3451 (was 3453) 2023-02-11 19:55:37,599 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=368, OpenFileDescriptor=747, MaxFileDescriptor=60000, SystemLoadAverage=138, ProcessCount=168, AvailableMemoryMB=3451 2023-02-11 19:55:37,599 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:37,601 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:55:37,602 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:55:37,603 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:37,604 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:55:37,605 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:55:37,606 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,606 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93 empty. 2023-02-11 19:55:37,607 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,607 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-11 19:55:37,615 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-11 19:55:37,616 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 63e0fe6a03fda2ed750f53c9d6195c93, NAME => 'testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:55:37,625 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:37,626 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 63e0fe6a03fda2ed750f53c9d6195c93, disabling compactions & flushes 2023-02-11 19:55:37,626 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:37,626 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:37,626 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. after waiting 0 ms 2023-02-11 19:55:37,626 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:37,626 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:37,626 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 63e0fe6a03fda2ed750f53c9d6195c93: Waiting for close lock at 1676145337626Disabling compacts and flushes for region at 1676145337626Disabling writes for close at 1676145337626Writing region close event to WAL at 1676145337626Closed at 1676145337626 2023-02-11 19:55:37,627 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:55:37,628 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145337628"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145337628"}]},"ts":"1676145337628"} 2023-02-11 19:55:37,629 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:55:37,630 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:55:37,630 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145337630"}]},"ts":"1676145337630"} 2023-02-11 19:55:37,631 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-11 19:55:37,633 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:55:37,634 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:55:37,634 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:55:37,634 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:55:37,634 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:55:37,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, ASSIGN}] 2023-02-11 19:55:37,635 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, ASSIGN 2023-02-11 19:55:37,636 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43747,1676145241852; forceNewPlan=false, retain=false 2023-02-11 19:55:37,786 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:55:37,787 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=63e0fe6a03fda2ed750f53c9d6195c93, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,787 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145337787"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145337787"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145337787"}]},"ts":"1676145337787"} 2023-02-11 19:55:37,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:55:37,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:37,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 63e0fe6a03fda2ed750f53c9d6195c93, NAME => 'testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:55:37,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:37,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:37,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,946 INFO [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,947 DEBUG [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1 2023-02-11 19:55:37,947 DEBUG [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1 2023-02-11 19:55:37,948 INFO [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63e0fe6a03fda2ed750f53c9d6195c93 columnFamilyName cf1 2023-02-11 19:55:37,948 INFO [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] regionserver.HStore(310): Store=63e0fe6a03fda2ed750f53c9d6195c93/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:37,949 INFO [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,950 DEBUG [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2 2023-02-11 19:55:37,950 DEBUG [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2 2023-02-11 19:55:37,950 INFO [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63e0fe6a03fda2ed750f53c9d6195c93 columnFamilyName cf2 2023-02-11 19:55:37,950 INFO [StoreOpener-63e0fe6a03fda2ed750f53c9d6195c93-1] regionserver.HStore(310): Store=63e0fe6a03fda2ed750f53c9d6195c93/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:37,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:37,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:37,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:55:37,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 63e0fe6a03fda2ed750f53c9d6195c93; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71706244, jitterRate=0.06850630044937134}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:37,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 63e0fe6a03fda2ed750f53c9d6195c93: Running coprocessor pre-open hook at 1676145337945Writing region info on filesystem at 1676145337945Initializing all the Stores at 1676145337946 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145337946Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145337946Cleaning up temporary data from old regions at 1676145337952 (+6 ms)Cleaning up detritus from prior splits at 1676145337952Running coprocessor post-open hooks at 1676145337956 (+4 ms)Region opened successfully at 1676145337956 2023-02-11 19:55:37,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93., pid=111, masterSystemTime=1676145337940 2023-02-11 19:55:37,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:37,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:37,958 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=63e0fe6a03fda2ed750f53c9d6195c93, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:37,958 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145337958"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145337958"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145337958"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145337958"}]},"ts":"1676145337958"} 2023-02-11 19:55:37,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-11 19:55:37,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 in 171 msec 2023-02-11 19:55:37,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-11 19:55:37,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, ASSIGN in 327 msec 2023-02-11 19:55:37,963 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:55:37,963 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145337963"}]},"ts":"1676145337963"} 2023-02-11 19:55:37,964 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-11 19:55:37,967 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:55:37,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 365 msec 2023-02-11 19:55:38,110 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:38,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 19:55:38,111 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,111 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,111 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 19:55:38,111 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:55:38,112 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,112 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,113 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,113 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,113 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,113 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,113 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,113 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:38,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:38,115 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:38,115 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,115 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. started... 2023-02-11 19:55:38,118 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63e0fe6a03fda2ed750f53c9d6195c93 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 19:55:38,118 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,120 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,120 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,120 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,120 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,120 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,127 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/9cb57c2ba42549bf8535dc76b7491aae 2023-02-11 19:55:38,137 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/49d8be0396c5492bb0c2bef44afe9fca 2023-02-11 19:55:38,140 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/9cb57c2ba42549bf8535dc76b7491aae as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae 2023-02-11 19:55:38,143 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:38,144 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/49d8be0396c5492bb0c2bef44afe9fca as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca 2023-02-11 19:55:38,147 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:38,148 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 63e0fe6a03fda2ed750f53c9d6195c93 in 30ms, sequenceid=6, compaction requested=false 2023-02-11 19:55:38,148 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,148 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63e0fe6a03fda2ed750f53c9d6195c93: 2023-02-11 19:55:38,149 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,152 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,152 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,153 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,153 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 19:55:38,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70d9b6e7[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:38,155 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:55:38,155 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,157 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:38,157 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:38,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,157 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:38,159 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,159 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,159 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,159 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,159 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,159 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,162 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,162 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,162 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,162 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:55:38,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:55:38,165 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 19:55:38,165 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:38,165 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:38,165 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,165 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:38,167 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,167 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,167 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,167 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,167 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,167 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,167 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,168 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,168 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,169 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,170 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,170 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,170 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,173 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,173 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,174 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:38,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,173 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:38,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,174 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,274 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:38,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:38,284 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:38,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:38,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 19:55:38,286 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,286 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,286 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 19:55:38,286 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:55:38,286 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,286 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,288 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,288 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,288 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,288 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,288 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,288 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:38,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,289 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,290 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,290 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,290 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:38,290 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:38,290 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,290 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. started... 2023-02-11 19:55:38,290 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63e0fe6a03fda2ed750f53c9d6195c93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:38,292 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,293 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,293 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,293 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,293 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,293 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,298 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/eb577a6c86084cdf896e2f966209de67 2023-02-11 19:55:38,308 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/6c9abb7f60dd4a55b23b1f84987411f4 2023-02-11 19:55:38,311 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/eb577a6c86084cdf896e2f966209de67 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67 2023-02-11 19:55:38,314 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:55:38,315 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/6c9abb7f60dd4a55b23b1f84987411f4 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4 2023-02-11 19:55:38,318 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:55:38,319 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63e0fe6a03fda2ed750f53c9d6195c93 in 28ms, sequenceid=19, compaction requested=false 2023-02-11 19:55:38,319 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63e0fe6a03fda2ed750f53c9d6195c93: 2023-02-11 19:55:38,319 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,319 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:38,319 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:38,319 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,319 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,319 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,322 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,322 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,323 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,323 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 19:55:38,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@297ae03d[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:38,324 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:55:38,324 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,326 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:38,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:38,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:38,326 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:38,327 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,327 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,327 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,327 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,329 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,329 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,329 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,330 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,330 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,330 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:55:38,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:55:38,333 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 19:55:38,333 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:38,333 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:38,333 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,333 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:38,335 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,335 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,335 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,335 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,336 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,337 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,338 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,338 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,340 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,341 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,340 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,341 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:38,340 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,341 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,341 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,341 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,341 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,341 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:38,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,341 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:38,441 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:38,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:38,453 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:38,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:38,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 19:55:38,454 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,454 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,454 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 19:55:38,454 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:55:38,455 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,455 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,457 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,457 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,457 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,457 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,458 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,458 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,459 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,459 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,460 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:38,460 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:38,460 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,460 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. started... 2023-02-11 19:55:38,460 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63e0fe6a03fda2ed750f53c9d6195c93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:38,461 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,461 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,461 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,462 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,462 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,462 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,468 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/68925bfdb62a45b082dae55f31b20103 2023-02-11 19:55:38,483 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/2c1de4b586f54aa7bb7b07f976a14d98 2023-02-11 19:55:38,487 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/68925bfdb62a45b082dae55f31b20103 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103 2023-02-11 19:55:38,491 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:55:38,491 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/2c1de4b586f54aa7bb7b07f976a14d98 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98 2023-02-11 19:55:38,495 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:55:38,495 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63e0fe6a03fda2ed750f53c9d6195c93 in 35ms, sequenceid=32, compaction requested=true 2023-02-11 19:55:38,496 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63e0fe6a03fda2ed750f53c9d6195c93: 2023-02-11 19:55:38,496 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,496 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:38,496 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:38,496 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,496 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,496 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,497 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,497 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,498 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,498 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 19:55:38,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64d3773d[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:38,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:55:38,500 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,501 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,501 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,501 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,502 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,502 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:38,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:38,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:38,502 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:38,504 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,504 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,504 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,504 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,504 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,504 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,505 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,505 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,505 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,505 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:55:38,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:55:38,508 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 19:55:38,509 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:38,509 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:38,509 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,509 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:38,511 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,511 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,511 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,512 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,512 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,512 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,514 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,520 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,520 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,520 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,521 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:38,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution because node does not exist (not an error) 2023-02-11 19:55:38,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:38,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,521 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,522 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,522 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:38,622 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:38,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:38,633 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:38,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:38,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 19:55:38,634 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,634 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,634 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 19:55:38,634 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:55:38,635 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,635 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,636 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,636 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,636 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,636 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,637 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:38,638 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:38,638 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,638 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. started... 2023-02-11 19:55:38,638 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63e0fe6a03fda2ed750f53c9d6195c93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:38,639 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,641 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,641 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,641 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,641 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,641 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,646 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/3d30ffda7faa4a5aa69068a1843c9d46 2023-02-11 19:55:38,656 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/6070baa23fc5442d833d83f2bcc13139 2023-02-11 19:55:38,659 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/3d30ffda7faa4a5aa69068a1843c9d46 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46 2023-02-11 19:55:38,662 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:55:38,663 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/6070baa23fc5442d833d83f2bcc13139 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139 2023-02-11 19:55:38,666 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:55:38,667 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63e0fe6a03fda2ed750f53c9d6195c93 in 29ms, sequenceid=45, compaction requested=true 2023-02-11 19:55:38,667 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63e0fe6a03fda2ed750f53c9d6195c93: 2023-02-11 19:55:38,667 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,667 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:38,667 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:38,667 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,667 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,667 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,669 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,669 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,669 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,669 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 19:55:38,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@440e833d[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:38,671 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:55:38,671 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,672 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,672 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,673 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,673 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:38,673 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,673 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,673 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:38,672 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,673 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:38,673 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,673 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,673 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:38,674 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,674 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,674 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,674 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,676 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,676 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,676 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,676 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,676 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,676 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:55:38,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:55:38,679 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 19:55:38,679 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:38,679 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:38,679 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,679 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:38,680 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,680 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,680 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,680 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,680 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,680 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,680 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,681 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,681 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,682 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,683 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,683 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,684 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,686 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,686 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,686 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,687 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:38,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:38,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,687 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:38,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,788 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:38,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:38,798 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:38,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:38,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 19:55:38,800 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,800 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,800 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 19:55:38,800 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:55:38,800 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,800 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,802 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,802 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,802 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,802 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,803 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,803 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:38,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,804 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,805 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,805 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,805 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:38,805 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:38,805 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,805 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. started... 2023-02-11 19:55:38,805 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63e0fe6a03fda2ed750f53c9d6195c93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:38,806 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,807 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,807 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,807 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,807 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,807 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,813 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/b42cf5177be541cdb300d0e521844e90 2023-02-11 19:55:38,824 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/df12545569844869840e736280cc2439 2023-02-11 19:55:38,829 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/b42cf5177be541cdb300d0e521844e90 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90 2023-02-11 19:55:38,832 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:55:38,833 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/df12545569844869840e736280cc2439 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439 2023-02-11 19:55:38,836 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:55:38,837 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63e0fe6a03fda2ed750f53c9d6195c93 in 32ms, sequenceid=58, compaction requested=true 2023-02-11 19:55:38,837 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63e0fe6a03fda2ed750f53c9d6195c93: 2023-02-11 19:55:38,837 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,837 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,843 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,843 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,843 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,843 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 19:55:38,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79dd7bd5[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:38,845 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:55:38,845 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:38,846 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,846 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,848 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:55:38,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:55:38,851 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 19:55:38,851 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:38,851 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:38,851 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,851 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:38,855 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:38,855 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,856 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,855 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,856 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,856 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,858 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,858 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:38,858 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,860 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,861 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,860 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,861 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,861 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:38,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:38,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:38,861 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:38,862 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,862 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:38,862 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:38,862 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,862 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,962 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:38,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:38,973 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:38,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:38,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 19:55:38,974 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:38,974 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,974 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 19:55:38,974 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 19:55:38,975 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,975 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,976 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,976 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,976 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,976 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:38,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:38,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:38,977 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:38,977 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 19:55:38,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 19:55:38,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:38,978 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:38,978 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:38,978 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. started... 2023-02-11 19:55:38,978 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63e0fe6a03fda2ed750f53c9d6195c93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:38,979 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,981 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,981 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,981 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,981 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:38,981 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:38,986 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a 2023-02-11 19:55:38,996 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/79dfd288d01146d2963f7b22b8d95e3a 2023-02-11 19:55:38,999 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a 2023-02-11 19:55:39,003 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:55:39,004 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/79dfd288d01146d2963f7b22b8d95e3a as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a 2023-02-11 19:55:39,007 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:55:39,008 INFO [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63e0fe6a03fda2ed750f53c9d6195c93 in 30ms, sequenceid=71, compaction requested=true 2023-02-11 19:55:39,008 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63e0fe6a03fda2ed750f53c9d6195c93: 2023-02-11 19:55:39,008 DEBUG [rs(jenkins-hbase4.apache.org,43747,1676145241852)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:39,008 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:39,008 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:39,008 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,008 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 19:55:39,008 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 19:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,010 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:39,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 19:55:39,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75807db[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:39,012 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 19:55:39,012 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:39,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:39,013 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,013 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:39,013 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:39,015 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:39,015 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:39,015 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:39,016 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 19:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:39,016 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:39,017 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 19:55:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:39,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,019 ERROR [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 1c013e61-8b4b-4f0a-8efd-0dfb5b64fb4d) service to localhost/127.0.0.1:42039 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:55:39,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43747,1676145241852': 2023-02-11 19:55:39,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 19:55:39,020 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 19:55:39,020 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:39,020 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:39,020 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 19:55:39,020 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:39,021 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,021 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,021 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:39,021 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,021 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,021 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,022 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:39,022 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:39,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:39,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,022 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:39,023 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:39,024 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 19:55:39,025 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,025 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:39,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:39,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,029 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:39,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:39,030 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 19:55:39,030 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:39,030 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:39,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:39,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:39,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:39,030 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,031 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,031 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:39,031 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:39,031 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,031 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 19:55:39,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:39,130 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:39,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:39,388 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 19:55:39,389 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:39,390 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,391 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,391 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,391 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,391 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:39,391 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:39,392 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:39,640 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,640 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:39,640 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:39,640 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:39,640 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-11 19:55:39,640 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:39,640 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:39,640 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:39,641 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:39,641 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:39,642 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 19:55:39,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:39,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:39,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:39,643 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:39,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:39,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:39,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:39,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:39,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:39,644 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:39,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:39,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:39,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:39,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:39,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:39,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:39,645 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:39,646 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:39,646 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:39,646 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:39,646 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:39,646 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:39,646 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:39,647 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:39,647 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:39,647 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:39,647 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:39,648 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:39,648 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:39,648 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:39,648 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:39,648 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:39,648 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:39,649 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:39,649 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,649 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:39,649 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,649 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,649 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,649 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 19:55:39,650 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:39,650 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:39,650 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145369650 2023-02-11 19:55:39,652 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:39,652 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,654 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:39,654 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:39,654 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:39,655 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145399655 2023-02-11 19:55:39,655 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145459655 2023-02-11 19:55:39,655 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:55:39,655 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:39,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:39,657 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:39,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:39,905 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,905 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:39,905 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:39,905 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:39,905 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:39,905 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:39,905 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:39,905 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:39,906 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:39,906 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:39,908 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 19:55:39,908 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:39,908 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:39,909 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:39,909 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:39,909 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:39,909 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:39,910 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:39,910 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:39,910 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:39,910 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:39,910 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:39,910 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:39,910 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:39,911 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:39,911 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:39,911 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:39,911 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:39,911 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:39,911 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:39,911 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:39,912 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:39,912 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:39,912 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:39,912 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:39,912 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:39,913 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:39,913 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:39,913 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:39,913 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:39,914 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:39,914 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:39,914 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:39,914 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:39,914 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:39,914 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,915 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:39,915 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,915 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,915 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,915 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-11 19:55:39,915 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:39,916 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:39,916 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145369916 2023-02-11 19:55:39,918 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:39,918 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,920 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:39,920 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:39,920 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:39,920 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145399920 2023-02-11 19:55:39,920 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145459920 2023-02-11 19:55:39,920 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:55:39,920 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:39,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN}] 2023-02-11 19:55:39,922 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:39,922 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:39,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:39,922 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:40,170 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,171 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:40,171 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:40,171 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:40,171 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:40,171 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:40,171 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:40,171 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:40,171 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:40,172 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:40,173 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 19:55:40,173 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:40,174 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:40,174 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:40,174 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:40,174 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:40,174 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:40,175 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:40,175 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:40,175 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:40,175 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:40,175 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:40,175 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:40,176 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:40,176 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:40,176 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:40,176 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:40,177 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:40,177 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:40,177 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:40,177 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:40,178 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:40,178 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:40,178 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:40,178 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:40,178 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:40,179 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:40,179 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:40,179 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:40,180 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:40,180 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:40,180 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:40,180 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:40,181 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:40,181 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:40,181 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,181 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:40,181 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,182 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,182 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,182 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-11 19:55:40,196 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:40,197 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:40,200 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145370200 2023-02-11 19:55:40,200 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:40,202 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,205 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:40,205 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:40,206 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:40,206 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145400206 2023-02-11 19:55:40,206 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145460206 2023-02-11 19:55:40,206 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 19:55:40,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN}] 2023-02-11 19:55:40,206 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:40,210 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:40,210 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:40,211 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145340210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145340210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145340210"}]},"ts":"1676145340210"} 2023-02-11 19:55:40,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:55:40,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 63e0fe6a03fda2ed750f53c9d6195c93, disabling compactions & flushes 2023-02-11 19:55:40,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:40,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:40,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. after waiting 0 ms 2023-02-11 19:55:40,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:40,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 63e0fe6a03fda2ed750f53c9d6195c93 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 19:55:40,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/eba50ab1e0c5404bad9a5d012bae5b38 2023-02-11 19:55:40,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/2f146641fc224ff6be9d494d68a081fa 2023-02-11 19:55:40,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf1/eba50ab1e0c5404bad9a5d012bae5b38 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38 2023-02-11 19:55:40,391 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:55:40,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/.tmp/cf2/2f146641fc224ff6be9d494d68a081fa as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa 2023-02-11 19:55:40,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:55:40,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 63e0fe6a03fda2ed750f53c9d6195c93 in 31ms, sequenceid=82, compaction requested=true 2023-02-11 19:55:40,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 19:55:40,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:40,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. 2023-02-11 19:55:40,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 63e0fe6a03fda2ed750f53c9d6195c93: Waiting for close lock at 1676145340365Running coprocessor pre-close hooks at 1676145340365Disabling compacts and flushes for region at 1676145340365Disabling writes for close at 1676145340365Obtaining lock to block concurrent updates at 1676145340365Preparing flush snapshotting stores in 63e0fe6a03fda2ed750f53c9d6195c93 at 1676145340365Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676145340365Flushing stores of testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. at 1676145340366 (+1 ms)Flushing 63e0fe6a03fda2ed750f53c9d6195c93/cf1: creating writer at 1676145340367 (+1 ms)Flushing 63e0fe6a03fda2ed750f53c9d6195c93/cf1: appending metadata at 1676145340368 (+1 ms)Flushing 63e0fe6a03fda2ed750f53c9d6195c93/cf1: closing flushed file at 1676145340369 (+1 ms)Flushing 63e0fe6a03fda2ed750f53c9d6195c93/cf2: creating writer at 1676145340378 (+9 ms)Flushing 63e0fe6a03fda2ed750f53c9d6195c93/cf2: appending metadata at 1676145340379 (+1 ms)Flushing 63e0fe6a03fda2ed750f53c9d6195c93/cf2: closing flushed file at 1676145340379Flushing 63e0fe6a03fda2ed750f53c9d6195c93/cf1: reopening flushed file at 1676145340388 (+9 ms)Flushing 63e0fe6a03fda2ed750f53c9d6195c93/cf2: reopening flushed file at 1676145340392 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 63e0fe6a03fda2ed750f53c9d6195c93 in 31ms, sequenceid=82, compaction requested=true at 1676145340396 (+4 ms)Writing region close event to WAL at 1676145340397 (+1 ms)Running coprocessor post-close hooks at 1676145340399 (+2 ms)Closed at 1676145340400 (+1 ms) 2023-02-11 19:55:40,402 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,403 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED 2023-02-11 19:55:40,403 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145340403"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145340403"}]},"ts":"1676145340403"} 2023-02-11 19:55:40,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-11 19:55:40,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 in 192 msec 2023-02-11 19:55:40,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-11 19:55:40,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN in 200 msec 2023-02-11 19:55:40,409 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:40,409 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:40,409 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:40,456 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,456 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:40,456 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:40,456 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:40,456 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:40,457 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:40,457 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:40,457 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:40,457 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:40,457 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:40,459 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 19:55:40,459 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:55:40,459 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:55:40,460 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:40,460 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:40,460 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:40,460 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:40,461 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:40,461 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:40,461 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:40,461 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:40,462 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:40,462 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:40,462 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:40,462 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:40,462 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:40,463 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:40,463 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:40,463 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:40,463 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:40,464 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:40,464 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:40,464 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:40,464 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:40,464 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:40,465 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:40,465 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:40,465 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:40,465 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:40,466 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:40,466 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:40,466 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:40,466 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:40,466 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:40,467 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:40,467 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:40,467 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:40,467 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,468 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:40,468 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:40,468 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:40,468 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,468 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,468 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,468 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:40,468 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:40,469 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-11 19:55:40,469 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:40,470 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:40,470 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145370470 2023-02-11 19:55:40,471 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:40,471 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:40,474 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:40,474 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:40,474 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:40,474 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145400474 2023-02-11 19:55:40,474 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145460474 2023-02-11 19:55:40,474 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:55:40,476 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:40,485 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=63e0fe6a03fda2ed750f53c9d6195c93, threads=14 2023-02-11 19:55:40,485 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,485 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,485 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,486 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,486 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,486 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,486 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,486 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,487 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,490 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,490 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,490 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,490 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,491 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,664 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,664 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,672 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,672 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,676 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,676 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,686 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,695 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,700 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,700 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,701 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,703 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:40,851 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 19:55:41,039 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,096 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,096 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 63e0fe6a03fda2ed750f53c9d6195c93 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:55:41,124 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:41,124 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:41,124 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:41,224 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,225 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:41,225 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:41,225 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:41,225 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:41,225 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:41,225 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:41,225 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:41,225 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:41,226 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:41,227 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 19:55:41,227 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:55:41,227 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:55:41,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:41,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:41,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:41,228 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:41,229 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:41,229 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:41,229 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:41,229 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:41,229 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:41,230 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:41,230 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:41,230 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:41,230 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:41,230 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:41,230 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:41,231 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:41,231 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:41,231 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:41,231 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:41,231 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:41,232 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:41,232 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:41,232 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:41,232 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:41,232 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:41,233 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:41,233 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:41,233 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:41,233 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:41,233 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:41,233 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:41,233 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:41,233 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:41,234 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:41,234 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,234 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:41,234 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:41,234 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:41,234 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,234 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,234 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,235 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:41,235 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:41,235 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-11 19:55:41,235 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:41,236 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:41,236 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145371236 2023-02-11 19:55:41,238 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:41,238 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,240 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:41,240 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:41,240 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:41,240 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145401240 2023-02-11 19:55:41,240 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145461240 2023-02-11 19:55:41,240 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:55:41,249 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=63e0fe6a03fda2ed750f53c9d6195c93, threads=14 2023-02-11 19:55:41,249 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,249 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,249 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,252 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,254 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,255 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,316 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,318 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,319 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,325 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,327 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,327 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,333 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,336 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,338 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,338 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,340 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,342 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,342 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,703 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139 for region: 63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:41,703 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 63e0fe6a03fda2ed750f53c9d6195c93 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:55:41,732 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:41,737 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:55:41,738 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:55:41,738 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:41,738 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:41,738 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:41,741 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,741 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:41,741 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:41,741 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:41,741 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:41,741 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:41,741 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:41,741 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:41,741 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:41,742 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:41,743 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 19:55:41,743 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:55:41,743 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:55:41,743 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:41,744 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:41,744 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:41,744 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:41,744 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:41,744 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:41,745 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:41,745 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:41,745 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:41,745 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:41,745 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:41,745 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:41,745 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:41,746 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:41,746 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:41,746 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:41,746 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:41,746 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:41,746 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:41,746 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:41,746 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:41,747 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:41,747 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:41,747 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:41,747 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:41,747 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:41,747 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:41,747 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:41,748 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:41,748 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:41,748 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:41,748 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:41,748 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:41,748 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:41,748 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,748 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:41,749 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:41,749 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:41,749 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,749 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,749 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,749 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:41,749 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:41,749 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 19:55:41,750 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:41,750 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:41,750 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145371750 2023-02-11 19:55:41,750 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:41,752 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,754 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:41,754 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:41,754 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:41,754 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145401754 2023-02-11 19:55:41,754 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145461754 2023-02-11 19:55:41,754 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:55:41,756 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:41,758 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:41,758 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:41,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:41,758 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:42,004 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,005 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:42,005 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:42,005 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:42,005 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:42,005 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:42,005 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:42,005 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:42,005 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:42,006 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:42,007 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 19:55:42,007 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:55:42,007 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:55:42,007 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:42,007 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:42,008 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:42,008 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:42,008 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,008 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:42,008 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,008 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,008 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,009 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:42,009 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,009 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,009 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,009 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:42,009 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:42,009 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,009 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,010 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,011 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,011 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:42,011 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,011 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,011 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,011 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,011 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,012 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:42,012 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,012 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:42,012 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:42,012 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:42,012 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,012 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,012 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,012 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:42,012 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:42,012 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 19:55:42,013 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:42,013 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:42,014 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145372014 2023-02-11 19:55:42,015 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,015 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:42,018 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:42,018 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:42,018 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:42,018 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145402018 2023-02-11 19:55:42,018 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145462018 2023-02-11 19:55:42,018 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 19:55:42,018 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:42,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145342020"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676145342020"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676145342020"}]},"ts":"1676145342020"} 2023-02-11 19:55:42,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145342020"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145342020"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145342020"}]},"ts":"1676145342020"} 2023-02-11 19:55:42,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145342020"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145342020"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145342020"}]},"ts":"1676145342020"} 2023-02-11 19:55:42,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:42,022 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:42,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:42,067 ERROR [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 2f25de57-daa1-42aa-aa7e-60a3bc1a94d0) service to localhost/127.0.0.1:42039 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:55:42,268 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,268 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:42,269 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:42,269 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:42,269 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:42,269 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:42,269 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:42,269 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:42,269 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:42,270 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:42,271 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-11 19:55:42,271 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:55:42,271 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:55:42,271 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:42,272 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:42,272 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:42,272 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:42,273 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,273 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:42,273 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,273 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,273 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,274 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:42,274 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,274 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,274 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,275 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:42,275 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:42,275 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,275 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,275 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,275 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:42,275 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,276 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,276 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,276 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:42,276 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,276 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,276 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,277 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,277 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:42,277 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,277 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,278 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,278 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,278 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,278 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:42,278 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,278 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:42,278 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:42,278 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:42,278 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,278 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,279 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,279 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:42,279 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:42,279 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-11 19:55:42,279 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:42,280 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:42,280 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145372280 2023-02-11 19:55:42,280 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:42,282 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,284 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:42,284 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:42,284 INFO [Listener at localhost/38771] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43747,1676145241852, table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:42,284 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=bb109fe57f8f41a7f75ac18fec133480, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 19:55:42,284 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b3f4ba21a4a9449716b53ec64fab580d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 19:55:42,285 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:42,285 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145402285 2023-02-11 19:55:42,285 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145462285 2023-02-11 19:55:42,285 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:55:42,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676145342285"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676145342285"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676145342285"}]},"ts":"1676145342285"} 2023-02-11 19:55:42,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145342285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145342285"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145342285"}]},"ts":"1676145342285"} 2023-02-11 19:55:42,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145342285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145342285"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145342285"}]},"ts":"1676145342285"} 2023-02-11 19:55:42,287 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:42,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:42,289 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:42,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:42,535 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,535 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:42,535 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:42,535 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:42,535 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:42,535 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:42,535 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:42,536 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:42,536 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:42,536 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:42,537 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-11 19:55:42,537 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:55:42,537 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:55:42,538 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:42,538 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:42,538 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:42,538 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:42,538 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,539 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:42,539 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,539 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,539 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,539 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:42,539 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,539 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,539 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,540 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:42,540 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:42,540 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,540 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,540 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,540 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:42,540 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,540 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,541 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,541 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:42,541 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,541 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,541 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,541 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,541 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:42,541 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:42,542 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:42,543 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,543 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,543 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,543 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:42,543 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:42,543 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 19:55:42,543 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:42,544 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:42,544 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145372544 2023-02-11 19:55:42,544 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:42,546 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,548 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:42,548 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:42,548 INFO [Listener at localhost/38771] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43747,1676145241852, table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:42,548 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=bb109fe57f8f41a7f75ac18fec133480, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 19:55:42,549 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b3f4ba21a4a9449716b53ec64fab580d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 19:55:42,549 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:42,549 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145402549 2023-02-11 19:55:42,549 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145462549 2023-02-11 19:55:42,549 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 19:55:42,549 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:42,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, ASSIGN}] 2023-02-11 19:55:42,551 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:42,551 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:42,551 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:42,799 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,799 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:42,800 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:42,800 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:42,800 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:42,800 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:42,800 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:42,800 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:42,800 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:42,801 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:42,802 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-11 19:55:42,802 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:55:42,802 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:55:42,802 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:42,802 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:42,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:42,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:42,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:42,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:42,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,803 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:42,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:42,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:42,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:42,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,804 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:42,805 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,806 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,806 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:42,806 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,806 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:42,806 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,806 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,806 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:42,806 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:42,807 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,807 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:42,807 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:42,807 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:42,807 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,807 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,807 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,807 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:42,807 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:42,807 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 19:55:42,808 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:42,808 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:42,809 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145372809 2023-02-11 19:55:42,809 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:42,810 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:42,812 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:42,812 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:42,812 INFO [Listener at localhost/38771] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43747,1676145241852, table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:42,813 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=bb109fe57f8f41a7f75ac18fec133480, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 19:55:42,813 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b3f4ba21a4a9449716b53ec64fab580d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 19:55:42,813 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:42,813 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145402813 2023-02-11 19:55:42,813 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145462813 2023-02-11 19:55:42,813 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:55:42,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, ASSIGN}] 2023-02-11 19:55:42,813 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:42,815 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, ASSIGN 2023-02-11 19:55:42,815 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43747,1676145241852; forceNewPlan=false, retain=false 2023-02-11 19:55:42,816 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, ASSIGN 2023-02-11 19:55:42,816 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43747,1676145241852; forceNewPlan=false, retain=false 2023-02-11 19:55:42,966 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 19:55:42,967 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b3f4ba21a4a9449716b53ec64fab580d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:42,967 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145342967"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145342967"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145342967"}]},"ts":"1676145342967"} 2023-02-11 19:55:42,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure b3f4ba21a4a9449716b53ec64fab580d, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:55:42,971 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=bb109fe57f8f41a7f75ac18fec133480, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:42,971 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145342971"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145342971"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145342971"}]},"ts":"1676145342971"} 2023-02-11 19:55:42,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure bb109fe57f8f41a7f75ac18fec133480, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:55:43,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:43,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b3f4ba21a4a9449716b53ec64fab580d, NAME => 'testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 19:55:43,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:43,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:43,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,124 INFO [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,125 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1 2023-02-11 19:55:43,125 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1 2023-02-11 19:55:43,125 INFO [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3f4ba21a4a9449716b53ec64fab580d columnFamilyName cf1 2023-02-11 19:55:43,136 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46-top 2023-02-11 19:55:43,139 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103-top 2023-02-11 19:55:43,142 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a-top 2023-02-11 19:55:43,145 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae-top 2023-02-11 19:55:43,149 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90-top 2023-02-11 19:55:43,152 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67-top 2023-02-11 19:55:43,155 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38-top 2023-02-11 19:55:43,155 INFO [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(310): Store=b3f4ba21a4a9449716b53ec64fab580d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:43,155 INFO [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,156 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2 2023-02-11 19:55:43,156 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2 2023-02-11 19:55:43,157 INFO [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3f4ba21a4a9449716b53ec64fab580d columnFamilyName cf2 2023-02-11 19:55:43,177 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98-top 2023-02-11 19:55:43,182 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa-top 2023-02-11 19:55:43,196 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca-top 2023-02-11 19:55:43,201 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139-top 2023-02-11 19:55:43,207 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4-top 2023-02-11 19:55:43,211 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a-top 2023-02-11 19:55:43,214 DEBUG [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439-top 2023-02-11 19:55:43,215 INFO [StoreOpener-b3f4ba21a4a9449716b53ec64fab580d-1] regionserver.HStore(310): Store=b3f4ba21a4a9449716b53ec64fab580d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:43,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:43,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b3f4ba21a4a9449716b53ec64fab580d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64314747, jitterRate=-0.04163558781147003}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:43,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b3f4ba21a4a9449716b53ec64fab580d: Running coprocessor pre-open hook at 1676145343123Writing region info on filesystem at 1676145343123Initializing all the Stores at 1676145343124 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145343124Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145343124Cleaning up temporary data from old regions at 1676145343217 (+93 ms)Cleaning up detritus from prior splits at 1676145343218 (+1 ms)Running coprocessor post-open hooks at 1676145343221 (+3 ms)Region opened successfully at 1676145343221 2023-02-11 19:55:43,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d., pid=117, masterSystemTime=1676145343120 2023-02-11 19:55:43,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:43,223 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:43,223 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:43,223 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:43,223 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:43,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:55:43,223 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1912): b3f4ba21a4a9449716b53ec64fab580d/cf2 is initiating minor compaction (all files) 2023-02-11 19:55:43,223 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1912): b3f4ba21a4a9449716b53ec64fab580d/cf1 is initiating minor compaction (all files) 2023-02-11 19:55:43,223 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b3f4ba21a4a9449716b53ec64fab580d/cf2 in testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:43,223 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b3f4ba21a4a9449716b53ec64fab580d/cf1 in testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:43,224 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa-top] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/.tmp, totalSize=34.9 K 2023-02-11 19:55:43,224 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38-top] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/.tmp, totalSize=34.9 K 2023-02-11 19:55:43,225 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145338106 2023-02-11 19:55:43,225 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145338106 2023-02-11 19:55:43,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:43,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:43,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:43,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bb109fe57f8f41a7f75ac18fec133480, NAME => 'testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 19:55:43,225 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145338275 2023-02-11 19:55:43,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b3f4ba21a4a9449716b53ec64fab580d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:43,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:43,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145343225"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145343225"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145343225"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145343225"}]},"ts":"1676145343225"} 2023-02-11 19:55:43,226 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145338275 2023-02-11 19:55:43,226 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145338443 2023-02-11 19:55:43,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:43,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:43,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:43,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:43,226 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145338443 2023-02-11 19:55:43,227 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676145338624 2023-02-11 19:55:43,227 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676145338624 2023-02-11 19:55:43,228 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676145338789 2023-02-11 19:55:43,228 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676145338789 2023-02-11 19:55:43,228 INFO [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:43,228 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676145338963 2023-02-11 19:55:43,229 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676145338963 2023-02-11 19:55:43,229 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145339132 2023-02-11 19:55:43,229 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1 2023-02-11 19:55:43,229 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145339132 2023-02-11 19:55:43,229 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1 2023-02-11 19:55:43,231 INFO [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb109fe57f8f41a7f75ac18fec133480 columnFamilyName cf1 2023-02-11 19:55:43,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-11 19:55:43,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b3f4ba21a4a9449716b53ec64fab580d, server=jenkins-hbase4.apache.org,43747,1676145241852 in 258 msec 2023-02-11 19:55:43,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, ASSIGN in 423 msec 2023-02-11 19:55:43,256 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] throttle.PressureAwareThroughputController(145): b3f4ba21a4a9449716b53ec64fab580d#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:43,257 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] throttle.PressureAwareThroughputController(145): b3f4ba21a4a9449716b53ec64fab580d#cf2#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:43,270 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46-bottom 2023-02-11 19:55:43,276 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103-bottom 2023-02-11 19:55:43,280 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a-bottom 2023-02-11 19:55:43,282 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/.tmp/cf1/21b83be188014209b61597d82a8f7201 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/21b83be188014209b61597d82a8f7201 2023-02-11 19:55:43,283 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae-bottom 2023-02-11 19:55:43,284 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/.tmp/cf2/2a60ef2b2b014b86ace7e0b0edb058f2 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2a60ef2b2b014b86ace7e0b0edb058f2 2023-02-11 19:55:43,287 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90-bottom 2023-02-11 19:55:43,289 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b3f4ba21a4a9449716b53ec64fab580d/cf2 of b3f4ba21a4a9449716b53ec64fab580d into 2a60ef2b2b014b86ace7e0b0edb058f2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:43,289 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b3f4ba21a4a9449716b53ec64fab580d: 2023-02-11 19:55:43,289 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d., storeName=b3f4ba21a4a9449716b53ec64fab580d/cf2, priority=9, startTime=1676145343223; duration=0sec 2023-02-11 19:55:43,289 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:43,291 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67-bottom 2023-02-11 19:55:43,291 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b3f4ba21a4a9449716b53ec64fab580d/cf1 of b3f4ba21a4a9449716b53ec64fab580d into 21b83be188014209b61597d82a8f7201(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:43,291 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b3f4ba21a4a9449716b53ec64fab580d: 2023-02-11 19:55:43,291 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d., storeName=b3f4ba21a4a9449716b53ec64fab580d/cf1, priority=9, startTime=1676145343222; duration=0sec 2023-02-11 19:55:43,291 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:43,295 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38-bottom 2023-02-11 19:55:43,296 INFO [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(310): Store=bb109fe57f8f41a7f75ac18fec133480/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:43,296 INFO [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:43,297 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2 2023-02-11 19:55:43,297 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2 2023-02-11 19:55:43,297 INFO [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb109fe57f8f41a7f75ac18fec133480 columnFamilyName cf2 2023-02-11 19:55:43,316 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98-bottom 2023-02-11 19:55:43,320 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa-bottom 2023-02-11 19:55:43,323 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca-bottom 2023-02-11 19:55:43,327 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139-bottom 2023-02-11 19:55:43,330 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4-bottom 2023-02-11 19:55:43,338 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a-bottom 2023-02-11 19:55:43,341 DEBUG [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439-bottom 2023-02-11 19:55:43,341 INFO [StoreOpener-bb109fe57f8f41a7f75ac18fec133480-1] regionserver.HStore(310): Store=bb109fe57f8f41a7f75ac18fec133480/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:43,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:43,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:43,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:43,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:43,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bb109fe57f8f41a7f75ac18fec133480; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61681772, jitterRate=-0.08086997270584106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:43,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bb109fe57f8f41a7f75ac18fec133480: Running coprocessor pre-open hook at 1676145343227Writing region info on filesystem at 1676145343227Initializing all the Stores at 1676145343228 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145343228Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145343228Cleaning up temporary data from old regions at 1676145343343 (+115 ms)Cleaning up detritus from prior splits at 1676145343344 (+1 ms)Running coprocessor post-open hooks at 1676145343347 (+3 ms)Region opened successfully at 1676145343347 2023-02-11 19:55:43,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480., pid=118, masterSystemTime=1676145343120 2023-02-11 19:55:43,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:43,349 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:43,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:55:43,349 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:43,349 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:43,349 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1912): bb109fe57f8f41a7f75ac18fec133480/cf1 is initiating minor compaction (all files) 2023-02-11 19:55:43,349 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:43,349 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1912): bb109fe57f8f41a7f75ac18fec133480/cf2 is initiating minor compaction (all files) 2023-02-11 19:55:43,349 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bb109fe57f8f41a7f75ac18fec133480/cf1 in testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:43,349 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegion(2258): Starting compaction of bb109fe57f8f41a7f75ac18fec133480/cf2 in testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:43,349 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/.tmp, totalSize=34.9 K 2023-02-11 19:55:43,349 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/.tmp, totalSize=34.9 K 2023-02-11 19:55:43,350 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145338106 2023-02-11 19:55:43,350 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145338106 2023-02-11 19:55:43,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:43,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:43,351 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=bb109fe57f8f41a7f75ac18fec133480, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:43,351 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145343351"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145343351"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145343351"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145343351"}]},"ts":"1676145343351"} 2023-02-11 19:55:43,352 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145338275 2023-02-11 19:55:43,352 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145338275 2023-02-11 19:55:43,352 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145338443 2023-02-11 19:55:43,352 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145338443 2023-02-11 19:55:43,353 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145338624 2023-02-11 19:55:43,353 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145338624 2023-02-11 19:55:43,353 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145338789 2023-02-11 19:55:43,353 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145338789 2023-02-11 19:55:43,354 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145338963 2023-02-11 19:55:43,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-11 19:55:43,354 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting 6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145338963 2023-02-11 19:55:43,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure bb109fe57f8f41a7f75ac18fec133480, server=jenkins-hbase4.apache.org,43747,1676145241852 in 380 msec 2023-02-11 19:55:43,354 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] compactions.Compactor(207): Compacting 2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145339132 2023-02-11 19:55:43,354 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] compactions.Compactor(207): Compacting eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145339132 2023-02-11 19:55:43,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-11 19:55:43,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, ASSIGN in 541 msec 2023-02-11 19:55:43,357 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 19:55:43,357 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:43,357 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 19:55:43,366 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] throttle.PressureAwareThroughputController(145): bb109fe57f8f41a7f75ac18fec133480#cf1#compaction#111 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:43,366 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] throttle.PressureAwareThroughputController(145): bb109fe57f8f41a7f75ac18fec133480#cf2#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:43,382 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/.tmp/cf2/26967da455bb4f2da97ef3e908237863 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/26967da455bb4f2da97ef3e908237863 2023-02-11 19:55:43,382 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/.tmp/cf1/d445022c2b6144089573d95770b83198 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/d445022c2b6144089573d95770b83198 2023-02-11 19:55:43,386 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bb109fe57f8f41a7f75ac18fec133480/cf1 of bb109fe57f8f41a7f75ac18fec133480 into d445022c2b6144089573d95770b83198(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:43,386 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bb109fe57f8f41a7f75ac18fec133480/cf2 of bb109fe57f8f41a7f75ac18fec133480 into 26967da455bb4f2da97ef3e908237863(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:43,386 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bb109fe57f8f41a7f75ac18fec133480: 2023-02-11 19:55:43,386 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for bb109fe57f8f41a7f75ac18fec133480: 2023-02-11 19:55:43,386 INFO [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480., storeName=bb109fe57f8f41a7f75ac18fec133480/cf1, priority=9, startTime=1676145343348; duration=0sec 2023-02-11 19:55:43,386 INFO [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480., storeName=bb109fe57f8f41a7f75ac18fec133480/cf2, priority=9, startTime=1676145343349; duration=0sec 2023-02-11 19:55:43,386 DEBUG [RS:1;jenkins-hbase4:43747-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:43,386 DEBUG [RS:1;jenkins-hbase4:43747-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:43,563 INFO [Listener at localhost/38771] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,563 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 19:55:43,563 INFO [Listener at localhost/38771] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:43,563 INFO [Listener at localhost/38771] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:43,564 ERROR [Listener at localhost/38771] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 19:55:43,564 INFO [Listener at localhost/38771] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 19:55:43,564 INFO [Listener at localhost/38771] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 19:55:43,564 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 19:55:43,564 INFO [Listener at localhost/38771] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 19:55:43,565 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 19:55:43,566 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 19:55:43,567 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 19:55:43,567 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 19:55:43,567 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:43,567 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=23e1cc59de7125f5225460dcdfdb2b06, daughterA=740882a55424963b0009dd2ab104ea35, daughterB=53477071b15ad5d16d882e1728ca27a7 2023-02-11 19:55:43,567 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:43,567 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 19:55:43,567 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:43,568 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=281aa262152bf186516a3159abcf093d, daughterA=0f38e9ae28b3a3299ac12ff8d13dfd48, daughterB=4fdf3f42835ea1d9ba8f222165d44edd 2023-02-11 19:55:43,568 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:43,568 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 19:55:43,568 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:43,568 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b15981c436d314b9d2350f98152f693c, daughterA=7161492ed949257357e0dcca7cfffe17, daughterB=5a05c3396f111fa56b9bac0f72076513 2023-02-11 19:55:43,568 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b15981c436d314b9d2350f98152f693c, ASSIGN 2023-02-11 19:55:43,568 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:43,568 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=b5f5623d65d35fccbaee614c8f714736, daughterA=4bfdaf124bab3234a067d2fdb10d3655, daughterB=e5975d0156dee323074c22305f41a650 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8e9f8bd96b098a3d9b38f3d7bf6db2fe, daughterA=ed9dbe4102994d49d22b9d46d6f9665c, daughterB=0380d0a7ec1b73252e4007b78f1d4d8b 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:43,569 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:43,570 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42733,1676145241889, table=testRollbackForSplitTableRegionWithReplica, region=fcfe336b013ef46034d5376d55802014 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7ced170548c095f8fdf54b9ac1dd6465, daughterA=311b65ae393773ac95a00ea8f3c4c6bf, daughterB=82d6afa6bf7bec9457661390d8546e83 2023-02-11 19:55:43,570 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:43,570 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 19:55:43,570 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:43,570 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:43,570 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 19:55:43,570 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:43,571 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,571 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN 2023-02-11 19:55:43,571 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:43,571 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, ASSIGN 2023-02-11 19:55:43,571 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, ASSIGN 2023-02-11 19:55:43,571 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b3f4ba21a4a9449716b53ec64fab580d, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:43,572 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure bb109fe57f8f41a7f75ac18fec133480, server=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:43,572 DEBUG [Listener at localhost/38771] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d held the lock before restarting, call acquireLock to restore it. 2023-02-11 19:55:43,572 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,572 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,572 INFO [Listener at localhost/38771] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,572 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:43,572 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63e0fe6a03fda2ed750f53c9d6195c93, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:43,572 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:43,572 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:43,572 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b3f4ba21a4a9449716b53ec64fab580d, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:43,572 DEBUG [Listener at localhost/38771] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure bb109fe57f8f41a7f75ac18fec133480, server=jenkins-hbase4.apache.org,43747,1676145241852 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 19:55:43,572 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 19:55:43,573 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 19:55:43,573 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,32837,1676145241806, table=hbase:meta, region=1588230740 2023-02-11 19:55:43,574 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676145373574 2023-02-11 19:55:43,575 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:43,575 DEBUG [Listener at localhost/38771] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 19:55:43,579 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=6558cedf10894666d9fab2b1f6f4142d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42733,1676145241889, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889, openSeqNum=2 2023-02-11 19:55:43,579 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=63e0fe6a03fda2ed750f53c9d6195c93, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=2 2023-02-11 19:55:43,579 INFO [Listener at localhost/38771] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43747,1676145241852, table=testRecoveryAndDoubleExecution, region=63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:43,579 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=bb109fe57f8f41a7f75ac18fec133480, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=86 2023-02-11 19:55:43,579 DEBUG [Listener at localhost/38771] assignment.RegionStateStore(147): Load hbase:meta entry region=b3f4ba21a4a9449716b53ec64fab580d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43747,1676145241852, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852, openSeqNum=86 2023-02-11 19:55:43,579 INFO [Listener at localhost/38771] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 19:55:43,579 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676145403579 2023-02-11 19:55:43,579 INFO [Listener at localhost/38771] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676145463579 2023-02-11 19:55:43,579 INFO [Listener at localhost/38771] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 19:55:43,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 19:55:43,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63e0fe6a03fda2ed750f53c9d6195c93, daughterA=bb109fe57f8f41a7f75ac18fec133480, daughterB=b3f4ba21a4a9449716b53ec64fab580d in 4.1910 sec 2023-02-11 19:55:43,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43747] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:38392 deadline: 1676145403831, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93. is not online on jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:43,943 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 19:55:43,949 INFO [Listener at localhost/38771] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 19:55:43,951 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 19:55:43,958 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:43,960 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-11 19:55:43,960 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-11 19:55:43,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:43,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 19:55:43,963 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145343963"}]},"ts":"1676145343963"} 2023-02-11 19:55:43,964 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-11 19:55:43,966 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-11 19:55:43,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, UNASSIGN}] 2023-02-11 19:55:43,967 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, UNASSIGN 2023-02-11 19:55:43,967 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b3f4ba21a4a9449716b53ec64fab580d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:43,967 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145343967"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145343967"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145343967"}]},"ts":"1676145343967"} 2023-02-11 19:55:43,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure b3f4ba21a4a9449716b53ec64fab580d, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:55:43,970 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, UNASSIGN 2023-02-11 19:55:43,970 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=bb109fe57f8f41a7f75ac18fec133480, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:43,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145343970"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145343970"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145343970"}]},"ts":"1676145343970"} 2023-02-11 19:55:43,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure bb109fe57f8f41a7f75ac18fec133480, server=jenkins-hbase4.apache.org,43747,1676145241852}] 2023-02-11 19:55:44,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 19:55:44,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:44,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b3f4ba21a4a9449716b53ec64fab580d, disabling compactions & flushes 2023-02-11 19:55:44,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:44,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:44,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. after waiting 0 ms 2023-02-11 19:55:44,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:44,121 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38-top] to archive 2023-02-11 19:55:44,121 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:44,123 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,124 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,125 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,126 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,126 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,127 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,128 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,129 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa-top] to archive 2023-02-11 19:55:44,130 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:44,131 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,132 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,133 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,133 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,134 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,135 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,136 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:55:44,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:44,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d. 2023-02-11 19:55:44,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b3f4ba21a4a9449716b53ec64fab580d: Waiting for close lock at 1676145344120Running coprocessor pre-close hooks at 1676145344120Disabling compacts and flushes for region at 1676145344120Disabling writes for close at 1676145344120Writing region close event to WAL at 1676145344137 (+17 ms)Running coprocessor post-close hooks at 1676145344139 (+2 ms)Closed at 1676145344139 2023-02-11 19:55:44,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:44,142 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:44,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bb109fe57f8f41a7f75ac18fec133480, disabling compactions & flushes 2023-02-11 19:55:44,142 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:44,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:44,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. after waiting 0 ms 2023-02-11 19:55:44,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:44,142 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b3f4ba21a4a9449716b53ec64fab580d, regionState=CLOSED 2023-02-11 19:55:44,142 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145344142"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145344142"}]},"ts":"1676145344142"} 2023-02-11 19:55:44,143 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38-bottom] to archive 2023-02-11 19:55:44,143 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:44,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-11 19:55:44,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure b3f4ba21a4a9449716b53ec64fab580d, server=jenkins-hbase4.apache.org,43747,1676145241852 in 175 msec 2023-02-11 19:55:44,145 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b3f4ba21a4a9449716b53ec64fab580d, UNASSIGN in 179 msec 2023-02-11 19:55:44,146 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,147 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,148 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,149 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,150 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,150 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,152 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa-bottom] to archive 2023-02-11 19:55:44,152 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:44,153 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,154 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,155 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,156 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,157 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,158 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,159 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:55:44,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:44,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480. 2023-02-11 19:55:44,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bb109fe57f8f41a7f75ac18fec133480: Waiting for close lock at 1676145344142Running coprocessor pre-close hooks at 1676145344142Disabling compacts and flushes for region at 1676145344142Disabling writes for close at 1676145344142Writing region close event to WAL at 1676145344159 (+17 ms)Running coprocessor post-close hooks at 1676145344162 (+3 ms)Closed at 1676145344162 2023-02-11 19:55:44,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:44,164 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=bb109fe57f8f41a7f75ac18fec133480, regionState=CLOSED 2023-02-11 19:55:44,165 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676145344164"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145344164"}]},"ts":"1676145344164"} 2023-02-11 19:55:44,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-11 19:55:44,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure bb109fe57f8f41a7f75ac18fec133480, server=jenkins-hbase4.apache.org,43747,1676145241852 in 194 msec 2023-02-11 19:55:44,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-11 19:55:44,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb109fe57f8f41a7f75ac18fec133480, UNASSIGN in 201 msec 2023-02-11 19:55:44,169 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145344169"}]},"ts":"1676145344169"} 2023-02-11 19:55:44,170 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-11 19:55:44,171 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-11 19:55:44,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 212 msec 2023-02-11 19:55:44,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 19:55:44,265 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-11 19:55:44,266 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-11 19:55:44,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:44,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:44,269 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:44,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-11 19:55:44,273 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,273 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:44,273 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:44,274 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/recovered.edits] 2023-02-11 19:55:44,275 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/recovered.edits] 2023-02-11 19:55:44,275 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/recovered.edits] 2023-02-11 19:55:44,278 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/.regioninfo to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/.regioninfo 2023-02-11 19:55:44,278 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/3d30ffda7faa4a5aa69068a1843c9d46 2023-02-11 19:55:44,278 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/.regioninfo to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/.regioninfo 2023-02-11 19:55:44,279 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/68925bfdb62a45b082dae55f31b20103 2023-02-11 19:55:44,281 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a 2023-02-11 19:55:44,281 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,281 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/3d30ffda7faa4a5aa69068a1843c9d46.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,282 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/9cb57c2ba42549bf8535dc76b7491aae 2023-02-11 19:55:44,282 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,282 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/68925bfdb62a45b082dae55f31b20103.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,283 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/b42cf5177be541cdb300d0e521844e90 2023-02-11 19:55:44,283 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,283 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/6bfbd6ee46ee49b1bd6343edd6ba4b0a.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,284 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eb577a6c86084cdf896e2f966209de67 2023-02-11 19:55:44,284 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,285 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/9cb57c2ba42549bf8535dc76b7491aae.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,286 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,286 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf1/eba50ab1e0c5404bad9a5d012bae5b38 2023-02-11 19:55:44,286 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/b42cf5177be541cdb300d0e521844e90.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,287 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,287 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/eb577a6c86084cdf896e2f966209de67.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,289 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,289 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2c1de4b586f54aa7bb7b07f976a14d98 2023-02-11 19:55:44,289 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf1/eba50ab1e0c5404bad9a5d012bae5b38.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,290 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/2f146641fc224ff6be9d494d68a081fa 2023-02-11 19:55:44,296 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,296 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/49d8be0396c5492bb0c2bef44afe9fca 2023-02-11 19:55:44,296 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/2c1de4b586f54aa7bb7b07f976a14d98.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,297 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,299 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,300 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,301 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/2f146641fc224ff6be9d494d68a081fa.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,301 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6070baa23fc5442d833d83f2bcc13139 2023-02-11 19:55:44,302 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/49d8be0396c5492bb0c2bef44afe9fca.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,302 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,302 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/6c9abb7f60dd4a55b23b1f84987411f4 2023-02-11 19:55:44,304 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/79dfd288d01146d2963f7b22b8d95e3a 2023-02-11 19:55:44,304 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,304 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/6070baa23fc5442d833d83f2bcc13139.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,305 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/cf2/df12545569844869840e736280cc2439 2023-02-11 19:55:44,305 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/6c9abb7f60dd4a55b23b1f84987411f4.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,305 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/bb109fe57f8f41a7f75ac18fec133480/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,307 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/79dfd288d01146d2963f7b22b8d95e3a.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,308 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/d445022c2b6144089573d95770b83198 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf1/d445022c2b6144089573d95770b83198 2023-02-11 19:55:44,308 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/b3f4ba21a4a9449716b53ec64fab580d/cf2/df12545569844869840e736280cc2439.63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,309 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/recovered.edits/85.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93/recovered.edits/85.seqid 2023-02-11 19:55:44,309 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/63e0fe6a03fda2ed750f53c9d6195c93 2023-02-11 19:55:44,311 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/26967da455bb4f2da97ef3e908237863 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/cf2/26967da455bb4f2da97ef3e908237863 2023-02-11 19:55:44,311 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/21b83be188014209b61597d82a8f7201 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf1/21b83be188014209b61597d82a8f7201 2023-02-11 19:55:44,313 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2a60ef2b2b014b86ace7e0b0edb058f2 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/cf2/2a60ef2b2b014b86ace7e0b0edb058f2 2023-02-11 19:55:44,313 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480/recovered.edits/90.seqid 2023-02-11 19:55:44,314 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/bb109fe57f8f41a7f75ac18fec133480 2023-02-11 19:55:44,315 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d/recovered.edits/90.seqid 2023-02-11 19:55:44,316 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testRecoveryAndDoubleExecution/b3f4ba21a4a9449716b53ec64fab580d 2023-02-11 19:55:44,316 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-11 19:55:44,318 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:44,319 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-11 19:55:44,321 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-11 19:55:44,322 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:44,322 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-11 19:55:44,322 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145344322"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:44,322 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145344322"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:44,322 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145344322"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:44,323 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 19:55:44,323 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 63e0fe6a03fda2ed750f53c9d6195c93, NAME => 'testRecoveryAndDoubleExecution,,1676145337602.63e0fe6a03fda2ed750f53c9d6195c93.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => bb109fe57f8f41a7f75ac18fec133480, NAME => 'testRecoveryAndDoubleExecution,,1676145339389.bb109fe57f8f41a7f75ac18fec133480.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b3f4ba21a4a9449716b53ec64fab580d, NAME => 'testRecoveryAndDoubleExecution,41,1676145339389.b3f4ba21a4a9449716b53ec64fab580d.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 19:55:44,323 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-11 19:55:44,323 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145344323"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:44,325 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-11 19:55:44,326 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 19:55:44,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 60 msec 2023-02-11 19:55:44,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-11 19:55:44,370 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-11 19:55:44,385 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=374 (was 368) - Thread LEAK? -, OpenFileDescriptor=841 (was 747) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=135 (was 138), ProcessCount=168 (was 168), AvailableMemoryMB=3422 (was 3451) 2023-02-11 19:55:44,398 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=135, ProcessCount=168, AvailableMemoryMB=3421 2023-02-11 19:55:44,398 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:44,402 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:55:44,403 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:55:44,404 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,405 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:55:44,406 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:55:44,407 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,407 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e empty. 2023-02-11 19:55:44,408 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,408 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-11 19:55:44,417 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 19:55:44,418 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => b355cf38157b492859da4977d8fc798e, NAME => 'testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:55:44,425 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:44,425 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing b355cf38157b492859da4977d8fc798e, disabling compactions & flushes 2023-02-11 19:55:44,425 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:44,425 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:44,425 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. after waiting 0 ms 2023-02-11 19:55:44,425 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:44,425 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:44,425 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for b355cf38157b492859da4977d8fc798e: Waiting for close lock at 1676145344425Disabling compacts and flushes for region at 1676145344425Disabling writes for close at 1676145344425Writing region close event to WAL at 1676145344425Closed at 1676145344425 2023-02-11 19:55:44,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:55:44,428 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145344427"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145344427"}]},"ts":"1676145344427"} 2023-02-11 19:55:44,429 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:55:44,429 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:55:44,429 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145344429"}]},"ts":"1676145344429"} 2023-02-11 19:55:44,430 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-11 19:55:44,434 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:55:44,434 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:55:44,434 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:55:44,434 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:55:44,434 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:55:44,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b355cf38157b492859da4977d8fc798e, ASSIGN}] 2023-02-11 19:55:44,436 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b355cf38157b492859da4977d8fc798e, ASSIGN 2023-02-11 19:55:44,436 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b355cf38157b492859da4977d8fc798e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:55:44,586 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:55:44,587 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=b355cf38157b492859da4977d8fc798e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:44,588 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145344587"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145344587"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145344587"}]},"ts":"1676145344587"} 2023-02-11 19:55:44,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure b355cf38157b492859da4977d8fc798e, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:44,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:44,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b355cf38157b492859da4977d8fc798e, NAME => 'testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:55:44,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:44,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:44,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,747 INFO [StoreOpener-b355cf38157b492859da4977d8fc798e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,748 DEBUG [StoreOpener-b355cf38157b492859da4977d8fc798e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1 2023-02-11 19:55:44,748 DEBUG [StoreOpener-b355cf38157b492859da4977d8fc798e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1 2023-02-11 19:55:44,748 INFO [StoreOpener-b355cf38157b492859da4977d8fc798e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b355cf38157b492859da4977d8fc798e columnFamilyName cf1 2023-02-11 19:55:44,749 INFO [StoreOpener-b355cf38157b492859da4977d8fc798e-1] regionserver.HStore(310): Store=b355cf38157b492859da4977d8fc798e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:44,749 INFO [StoreOpener-b355cf38157b492859da4977d8fc798e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,750 DEBUG [StoreOpener-b355cf38157b492859da4977d8fc798e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2 2023-02-11 19:55:44,750 DEBUG [StoreOpener-b355cf38157b492859da4977d8fc798e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2 2023-02-11 19:55:44,750 INFO [StoreOpener-b355cf38157b492859da4977d8fc798e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b355cf38157b492859da4977d8fc798e columnFamilyName cf2 2023-02-11 19:55:44,750 INFO [StoreOpener-b355cf38157b492859da4977d8fc798e-1] regionserver.HStore(310): Store=b355cf38157b492859da4977d8fc798e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:44,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:44,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:44,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:55:44,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b355cf38157b492859da4977d8fc798e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69832324, jitterRate=0.04058271646499634}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:44,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b355cf38157b492859da4977d8fc798e: Running coprocessor pre-open hook at 1676145344746Writing region info on filesystem at 1676145344746Initializing all the Stores at 1676145344747 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145344747Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145344747Cleaning up temporary data from old regions at 1676145344752 (+5 ms)Cleaning up detritus from prior splits at 1676145344752Running coprocessor post-open hooks at 1676145344755 (+3 ms)Region opened successfully at 1676145344755 2023-02-11 19:55:44,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e., pid=127, masterSystemTime=1676145344740 2023-02-11 19:55:44,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:44,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:44,757 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=b355cf38157b492859da4977d8fc798e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:44,757 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145344757"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145344757"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145344757"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145344757"}]},"ts":"1676145344757"} 2023-02-11 19:55:44,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-11 19:55:44,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure b355cf38157b492859da4977d8fc798e, server=jenkins-hbase4.apache.org,42733,1676145241889 in 169 msec 2023-02-11 19:55:44,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-11 19:55:44,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b355cf38157b492859da4977d8fc798e, ASSIGN in 325 msec 2023-02-11 19:55:44,762 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:55:44,762 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145344762"}]},"ts":"1676145344762"} 2023-02-11 19:55:44,763 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-11 19:55:44,765 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:55:44,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 362 msec 2023-02-11 19:55:44,911 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:44,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,912 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:44,913 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:44,913 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 19:55:44,913 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 19:55:44,913 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,913 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,914 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:44,914 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:44,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:44,914 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:44,914 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:44,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:44,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:44,915 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:44,915 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,916 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:44,916 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:44,919 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,919 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,919 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,919 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,919 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:44,919 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:44,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:44,922 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:44,923 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:44,923 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:44,923 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:44,923 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:44,923 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:44,923 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. started... 2023-02-11 19:55:44,924 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b355cf38157b492859da4977d8fc798e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 19:55:45,015 ERROR [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 1d17abb3-77b3-4fd1-a223-1cc76825d5c0) service to localhost/127.0.0.1:42039 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:55:45,221 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:45,335 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/1f6cdd34396f4979a14aa180c4b7b903 2023-02-11 19:55:45,346 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/b746d6d0bac14cc18d77b58ec579fc6b 2023-02-11 19:55:45,350 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/1f6cdd34396f4979a14aa180c4b7b903 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/1f6cdd34396f4979a14aa180c4b7b903 2023-02-11 19:55:45,353 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/1f6cdd34396f4979a14aa180c4b7b903, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:45,354 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/b746d6d0bac14cc18d77b58ec579fc6b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/b746d6d0bac14cc18d77b58ec579fc6b 2023-02-11 19:55:45,356 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/b746d6d0bac14cc18d77b58ec579fc6b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:45,357 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b355cf38157b492859da4977d8fc798e in 433ms, sequenceid=6, compaction requested=false 2023-02-11 19:55:45,358 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b355cf38157b492859da4977d8fc798e: 2023-02-11 19:55:45,358 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:45,358 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:45,358 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:45,358 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,358 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:45,359 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:45,360 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,360 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:45,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:45,360 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,360 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:45,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:45,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:45,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:45,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 19:55:45,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78cde292[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:45,362 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 19:55:45,362 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,365 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:45,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:45,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:45,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:45,365 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:45,366 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:45,366 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:45,366 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:45,367 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:45,367 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:45,367 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:45,368 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:45,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:45,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:45,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:45,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:45,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:45,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:45,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:55:45,371 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 19:55:45,371 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:45,371 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:45,371 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,371 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:45,374 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:45,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:45,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,374 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,374 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,374 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:45,375 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,375 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:45,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,377 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:45,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,377 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,377 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:45,383 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:45,383 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:45,383 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:45,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,383 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,384 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:45,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:45,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,384 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,484 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:45,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:45,494 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:45,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:45,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,496 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:45,496 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:45,496 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 19:55:45,496 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 19:55:45,496 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,496 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,497 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,497 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,497 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,497 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:45,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,498 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:45,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:45,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:45,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:45,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:45,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:45,499 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:45,499 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:45,499 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. started... 2023-02-11 19:55:45,500 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b355cf38157b492859da4977d8fc798e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:45,501 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,502 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,502 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:45,502 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,502 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,502 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:45,901 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 19:55:45,907 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/61bb9f0b4a32498c97375c67e648e21e 2023-02-11 19:55:45,920 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c 2023-02-11 19:55:45,924 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/61bb9f0b4a32498c97375c67e648e21e as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/61bb9f0b4a32498c97375c67e648e21e 2023-02-11 19:55:45,928 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/61bb9f0b4a32498c97375c67e648e21e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:55:45,929 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c 2023-02-11 19:55:45,932 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:55:45,933 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b355cf38157b492859da4977d8fc798e in 433ms, sequenceid=19, compaction requested=false 2023-02-11 19:55:45,933 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b355cf38157b492859da4977d8fc798e: 2023-02-11 19:55:45,933 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:45,933 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:45,933 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:45,933 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,933 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:45,933 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:45,935 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,935 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:45,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:45,969 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,970 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:45,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:45,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:45,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:45,972 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 19:55:45,972 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@486a91dd[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:45,972 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 19:55:45,972 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:45,973 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,974 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:45,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:45,974 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:45,974 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:45,974 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:45,974 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:45,975 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:45,975 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:45,975 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:45,976 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:45,976 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:45,976 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:45,976 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:45,976 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,976 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:45,976 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:45,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:55:45,979 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 19:55:45,979 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:45,979 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:45,979 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,979 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:45,982 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:45,982 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,982 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,982 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:45,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:45,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:45,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,990 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,990 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,991 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,992 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:45,996 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,996 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:45,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:45,999 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:45,999 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:45,999 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:45,999 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:46,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:46,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,000 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,100 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:46,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:46,111 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:46,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:46,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,112 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:46,112 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:46,112 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 19:55:46,112 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 19:55:46,113 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,113 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,114 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,114 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,114 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,114 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,115 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,115 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:46,116 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:46,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,116 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,116 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,116 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:46,116 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,116 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:46,116 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,116 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:46,116 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:46,116 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:46,117 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:46,117 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:46,118 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:46,118 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:46,118 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:46,118 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:46,118 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:46,118 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:46,118 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. started... 2023-02-11 19:55:46,118 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b355cf38157b492859da4977d8fc798e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:46,119 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,121 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,121 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:46,121 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,121 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,121 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:46,136 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/b1260c0766c04aceb93c9dda593578cd 2023-02-11 19:55:46,549 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/06b8968eea244fbcae14f24a70890d41 2023-02-11 19:55:46,553 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/b1260c0766c04aceb93c9dda593578cd as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/b1260c0766c04aceb93c9dda593578cd 2023-02-11 19:55:46,556 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/b1260c0766c04aceb93c9dda593578cd, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:55:46,557 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/06b8968eea244fbcae14f24a70890d41 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/06b8968eea244fbcae14f24a70890d41 2023-02-11 19:55:46,560 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/06b8968eea244fbcae14f24a70890d41, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:55:46,560 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b355cf38157b492859da4977d8fc798e in 442ms, sequenceid=32, compaction requested=true 2023-02-11 19:55:46,560 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b355cf38157b492859da4977d8fc798e: 2023-02-11 19:55:46,560 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:46,560 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:46,562 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,562 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:46,562 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,562 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:46,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:46,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:46,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 19:55:46,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5afb58d6[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:46,564 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 19:55:46,564 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,565 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,565 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,565 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,565 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:46,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,566 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:46,566 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:46,566 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:46,566 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,566 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:46,566 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:46,566 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:46,566 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:46,566 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:46,566 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:46,575 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,575 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:46,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:46,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:46,575 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:46,575 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:46,576 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:46,576 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:46,576 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:46,577 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:46,577 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:46,577 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:46,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:55:46,579 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 19:55:46,579 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:46,579 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:46,579 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,579 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:46,580 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:46,581 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:46,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:46,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:46,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,581 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,582 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,582 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:46,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,583 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:46,584 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,584 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,587 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:46,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,587 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:46,588 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:46,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:46,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:46,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,588 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:46,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:46,587 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,588 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:46,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:46,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:46,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:46,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,588 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:46,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,688 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:46,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:46,699 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:46,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:46,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,700 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:46,700 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:46,700 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 19:55:46,700 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 19:55:46,701 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,701 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,702 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,702 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,702 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,702 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:46,702 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:46,702 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:46,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:46,704 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:46,704 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:46,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,705 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:46,706 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:46,706 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. started... 2023-02-11 19:55:46,706 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b355cf38157b492859da4977d8fc798e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:46,706 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,707 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,707 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,707 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:46,707 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:46,707 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:46,714 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/a1673d2d88a24364a8dad077cb11d3c1 2023-02-11 19:55:47,020 ERROR [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 1c013e61-8b4b-4f0a-8efd-0dfb5b64fb4d) service to localhost/127.0.0.1:42039 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:55:47,128 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/ebb7b6a00a624326b178ce2af9a2765e 2023-02-11 19:55:47,131 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/a1673d2d88a24364a8dad077cb11d3c1 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/a1673d2d88a24364a8dad077cb11d3c1 2023-02-11 19:55:47,134 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/a1673d2d88a24364a8dad077cb11d3c1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:55:47,135 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/ebb7b6a00a624326b178ce2af9a2765e as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/ebb7b6a00a624326b178ce2af9a2765e 2023-02-11 19:55:47,139 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/ebb7b6a00a624326b178ce2af9a2765e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:55:47,140 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b355cf38157b492859da4977d8fc798e in 434ms, sequenceid=45, compaction requested=true 2023-02-11 19:55:47,140 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b355cf38157b492859da4977d8fc798e: 2023-02-11 19:55:47,140 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:47,140 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:47,140 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:47,140 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,140 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:47,140 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:47,142 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,142 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,142 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,142 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:47,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 19:55:47,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ddaea8d[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:47,144 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 19:55:47,144 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,145 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,145 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:47,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:47,147 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,147 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,147 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,150 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,150 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,150 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,150 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,151 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,151 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,151 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:47,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:55:47,154 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 19:55:47,154 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:47,154 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:47,154 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,154 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:47,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,156 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,157 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,157 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,159 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,159 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,159 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,165 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,165 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:47,165 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:47,165 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,166 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:47,166 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,166 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,166 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,166 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,266 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:47,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:47,276 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:47,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:47,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,278 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:47,278 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:47,278 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 19:55:47,278 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 19:55:47,278 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,278 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,279 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,279 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,279 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,279 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,280 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,280 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,281 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:47,281 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:47,281 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:47,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:47,282 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:47,282 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. started... 2023-02-11 19:55:47,282 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b355cf38157b492859da4977d8fc798e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,285 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,285 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,285 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:47,285 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,285 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:47,290 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/f56fcfc7f14e42ecb7336b25c5b816fc 2023-02-11 19:55:47,302 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/7c083124d811411fb635f731086c8e2e 2023-02-11 19:55:47,306 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/f56fcfc7f14e42ecb7336b25c5b816fc as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/f56fcfc7f14e42ecb7336b25c5b816fc 2023-02-11 19:55:47,308 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/f56fcfc7f14e42ecb7336b25c5b816fc, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:55:47,309 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/7c083124d811411fb635f731086c8e2e as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/7c083124d811411fb635f731086c8e2e 2023-02-11 19:55:47,312 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/7c083124d811411fb635f731086c8e2e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:55:47,312 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b355cf38157b492859da4977d8fc798e in 30ms, sequenceid=58, compaction requested=true 2023-02-11 19:55:47,312 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b355cf38157b492859da4977d8fc798e: 2023-02-11 19:55:47,312 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:47,312 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:47,313 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:47,313 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,313 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:47,313 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:47,314 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,314 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,314 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,314 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:47,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 19:55:47,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a1442d6[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:47,316 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 19:55:47,316 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:47,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:47,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:47,318 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,318 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,318 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,318 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,320 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,320 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,320 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,320 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,320 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,321 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:47,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:55:47,323 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 19:55:47,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:47,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:47,323 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,323 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:47,324 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,324 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,325 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,325 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,326 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,326 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,328 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,328 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,328 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,331 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:47,331 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:47,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,331 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,330 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,331 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:47,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,331 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:47,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,331 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,332 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,332 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,332 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,332 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,332 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,332 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,431 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:47,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:47,442 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:47,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:47,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,444 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:47,444 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:47,444 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 19:55:47,444 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 19:55:47,445 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,445 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,446 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,446 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,446 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,446 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,446 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,446 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:47,447 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,447 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:47,447 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:47,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 19:55:47,448 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:47,448 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. started... 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:47,448 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:47,448 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b355cf38157b492859da4977d8fc798e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:47,450 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,450 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,451 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,451 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:47,451 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,451 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:47,857 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/72c50894d00c43be8e6a008ca54967e2 2023-02-11 19:55:47,867 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/954149af44924a4abfcebc19bd47ef44 2023-02-11 19:55:47,870 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/72c50894d00c43be8e6a008ca54967e2 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/72c50894d00c43be8e6a008ca54967e2 2023-02-11 19:55:47,872 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/72c50894d00c43be8e6a008ca54967e2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:55:47,873 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/954149af44924a4abfcebc19bd47ef44 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/954149af44924a4abfcebc19bd47ef44 2023-02-11 19:55:47,876 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/954149af44924a4abfcebc19bd47ef44, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:55:47,876 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b355cf38157b492859da4977d8fc798e in 428ms, sequenceid=71, compaction requested=true 2023-02-11 19:55:47,877 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b355cf38157b492859da4977d8fc798e: 2023-02-11 19:55:47,877 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:47,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:47,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:47,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 19:55:47,877 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 19:55:47,878 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,878 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,879 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,879 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:47,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,880 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 19:55:47,880 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@166cde30[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:47,880 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 19:55:47,880 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,884 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,884 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:47,884 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,884 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,885 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 19:55:47,885 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 19:55:47,885 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:47,885 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:47,885 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,887 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,887 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 19:55:47,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:47,890 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 19:55:47,890 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 19:55:47,890 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:47,890 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:47,890 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,890 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:47,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:47,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:47,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:47,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,892 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,892 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:47,893 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:47,894 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,894 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,895 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,897 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,897 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 19:55:47,897 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:47,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:47,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:47,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:47,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:47,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:47,897 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:47,898 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:47,898 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:47,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,898 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,898 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 19:55:47,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:47,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:47,998 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:47,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:48,005 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:48,006 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b355cf38157b492859da4977d8fc798e, daughterA=c2a02331cabd6ae065e4a409f788b22c, daughterB=17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:48,007 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b355cf38157b492859da4977d8fc798e, daughterA=c2a02331cabd6ae065e4a409f788b22c, daughterB=17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:48,007 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b355cf38157b492859da4977d8fc798e, daughterA=c2a02331cabd6ae065e4a409f788b22c, daughterB=17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:48,007 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b355cf38157b492859da4977d8fc798e, daughterA=c2a02331cabd6ae065e4a409f788b22c, daughterB=17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:48,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b355cf38157b492859da4977d8fc798e, UNASSIGN}] 2023-02-11 19:55:48,009 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b355cf38157b492859da4977d8fc798e, UNASSIGN 2023-02-11 19:55:48,009 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=b355cf38157b492859da4977d8fc798e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:48,009 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145348009"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145348009"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145348009"}]},"ts":"1676145348009"} 2023-02-11 19:55:48,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure b355cf38157b492859da4977d8fc798e, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:48,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b355cf38157b492859da4977d8fc798e, disabling compactions & flushes 2023-02-11 19:55:48,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:48,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:48,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. after waiting 0 ms 2023-02-11 19:55:48,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:48,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing b355cf38157b492859da4977d8fc798e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 19:55:48,172 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/321a850bea16494e9e803a7804012bc8 2023-02-11 19:55:48,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/0e20e087087a48648a61838965f7616b 2023-02-11 19:55:48,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf1/321a850bea16494e9e803a7804012bc8 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/321a850bea16494e9e803a7804012bc8 2023-02-11 19:55:48,187 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/321a850bea16494e9e803a7804012bc8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:55:48,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.tmp/cf2/0e20e087087a48648a61838965f7616b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/0e20e087087a48648a61838965f7616b 2023-02-11 19:55:48,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/0e20e087087a48648a61838965f7616b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:55:48,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b355cf38157b492859da4977d8fc798e in 29ms, sequenceid=82, compaction requested=true 2023-02-11 19:55:48,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 19:55:48,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:48,195 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. 2023-02-11 19:55:48,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b355cf38157b492859da4977d8fc798e: Waiting for close lock at 1676145348162Running coprocessor pre-close hooks at 1676145348162Disabling compacts and flushes for region at 1676145348162Disabling writes for close at 1676145348162Obtaining lock to block concurrent updates at 1676145348162Preparing flush snapshotting stores in b355cf38157b492859da4977d8fc798e at 1676145348162Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676145348162Flushing stores of testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. at 1676145348163 (+1 ms)Flushing b355cf38157b492859da4977d8fc798e/cf1: creating writer at 1676145348165 (+2 ms)Flushing b355cf38157b492859da4977d8fc798e/cf1: appending metadata at 1676145348167 (+2 ms)Flushing b355cf38157b492859da4977d8fc798e/cf1: closing flushed file at 1676145348167Flushing b355cf38157b492859da4977d8fc798e/cf2: creating writer at 1676145348175 (+8 ms)Flushing b355cf38157b492859da4977d8fc798e/cf2: appending metadata at 1676145348176 (+1 ms)Flushing b355cf38157b492859da4977d8fc798e/cf2: closing flushed file at 1676145348177 (+1 ms)Flushing b355cf38157b492859da4977d8fc798e/cf1: reopening flushed file at 1676145348185 (+8 ms)Flushing b355cf38157b492859da4977d8fc798e/cf2: reopening flushed file at 1676145348188 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b355cf38157b492859da4977d8fc798e in 29ms, sequenceid=82, compaction requested=true at 1676145348191 (+3 ms)Writing region close event to WAL at 1676145348192 (+1 ms)Running coprocessor post-close hooks at 1676145348195 (+3 ms)Closed at 1676145348195 2023-02-11 19:55:48,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,197 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=b355cf38157b492859da4977d8fc798e, regionState=CLOSED 2023-02-11 19:55:48,197 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145348197"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145348197"}]},"ts":"1676145348197"} 2023-02-11 19:55:48,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-11 19:55:48,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure b355cf38157b492859da4977d8fc798e, server=jenkins-hbase4.apache.org,42733,1676145241889 in 188 msec 2023-02-11 19:55:48,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-11 19:55:48,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b355cf38157b492859da4977d8fc798e, UNASSIGN in 192 msec 2023-02-11 19:55:48,211 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=b355cf38157b492859da4977d8fc798e, threads=14 2023-02-11 19:55:48,211 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/06b8968eea244fbcae14f24a70890d41 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,211 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/0e20e087087a48648a61838965f7616b for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,211 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,212 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/7c083124d811411fb635f731086c8e2e for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,212 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/954149af44924a4abfcebc19bd47ef44 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,212 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/b746d6d0bac14cc18d77b58ec579fc6b for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,212 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/ebb7b6a00a624326b178ce2af9a2765e for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,212 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/1f6cdd34396f4979a14aa180c4b7b903 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,216 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/321a850bea16494e9e803a7804012bc8 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,216 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/61bb9f0b4a32498c97375c67e648e21e for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,216 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/72c50894d00c43be8e6a008ca54967e2 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,216 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/a1673d2d88a24364a8dad077cb11d3c1 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,216 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/b1260c0766c04aceb93c9dda593578cd for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,219 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/f56fcfc7f14e42ecb7336b25c5b816fc for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,222 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/7c083124d811411fb635f731086c8e2e, top=false 2023-02-11 19:55:48,223 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/06b8968eea244fbcae14f24a70890d41, top=false 2023-02-11 19:55:48,223 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/954149af44924a4abfcebc19bd47ef44, top=false 2023-02-11 19:55:48,224 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/b746d6d0bac14cc18d77b58ec579fc6b, top=false 2023-02-11 19:55:48,227 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/ebb7b6a00a624326b178ce2af9a2765e, top=false 2023-02-11 19:55:48,227 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/72c50894d00c43be8e6a008ca54967e2, top=false 2023-02-11 19:55:48,228 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/b1260c0766c04aceb93c9dda593578cd, top=false 2023-02-11 19:55:48,228 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c, top=false 2023-02-11 19:55:48,230 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/61bb9f0b4a32498c97375c67e648e21e, top=false 2023-02-11 19:55:48,230 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/f56fcfc7f14e42ecb7336b25c5b816fc, top=false 2023-02-11 19:55:48,230 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/321a850bea16494e9e803a7804012bc8, top=false 2023-02-11 19:55:48,230 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/1f6cdd34396f4979a14aa180c4b7b903, top=false 2023-02-11 19:55:48,231 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/a1673d2d88a24364a8dad077cb11d3c1, top=false 2023-02-11 19:55:48,233 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/0e20e087087a48648a61838965f7616b, top=false 2023-02-11 19:55:48,252 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b1260c0766c04aceb93c9dda593578cd for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,253 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-ebb7b6a00a624326b178ce2af9a2765e for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,253 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-06b8968eea244fbcae14f24a70890d41 for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,253 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-f56fcfc7f14e42ecb7336b25c5b816fc for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,259 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-61bb9f0b4a32498c97375c67e648e21e for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,259 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-1f6cdd34396f4979a14aa180c4b7b903 for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,259 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-72c50894d00c43be8e6a008ca54967e2 for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,259 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-7c083124d811411fb635f731086c8e2e for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,261 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-0e20e087087a48648a61838965f7616b for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,261 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-954149af44924a4abfcebc19bd47ef44 for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,261 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-2233a757c6ab4a85b7d9e3d1e28d1f8c for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,264 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-321a850bea16494e9e803a7804012bc8 for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,264 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/b1260c0766c04aceb93c9dda593578cd for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,264 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/06b8968eea244fbcae14f24a70890d41 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,265 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/ebb7b6a00a624326b178ce2af9a2765e for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,266 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/72c50894d00c43be8e6a008ca54967e2 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,267 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/7c083124d811411fb635f731086c8e2e for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,268 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/1f6cdd34396f4979a14aa180c4b7b903 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,268 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/f56fcfc7f14e42ecb7336b25c5b816fc for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,268 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/0e20e087087a48648a61838965f7616b for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,268 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/61bb9f0b4a32498c97375c67e648e21e for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,271 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/954149af44924a4abfcebc19bd47ef44 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,272 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b746d6d0bac14cc18d77b58ec579fc6b for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,273 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/.splits/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-a1673d2d88a24364a8dad077cb11d3c1 for child: c2a02331cabd6ae065e4a409f788b22c, parent: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,274 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,274 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/321a850bea16494e9e803a7804012bc8 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,275 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/b746d6d0bac14cc18d77b58ec579fc6b for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,276 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/a1673d2d88a24364a8dad077cb11d3c1 for region: b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:48,276 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region b355cf38157b492859da4977d8fc798e Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-11 19:55:49,106 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:55:49,107 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:55:49,108 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676145349108"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676145349108"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676145349108"}]},"ts":"1676145349108"} 2023-02-11 19:55:49,109 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349108"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145349108"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145349108"}]},"ts":"1676145349108"} 2023-02-11 19:55:49,109 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349108"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145349108"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145349108"}]},"ts":"1676145349108"} 2023-02-11 19:55:49,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c2a02331cabd6ae065e4a409f788b22c, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=17cfe364c2325fc5b7f2356e26a73a8e, ASSIGN}] 2023-02-11 19:55:49,111 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=17cfe364c2325fc5b7f2356e26a73a8e, ASSIGN 2023-02-11 19:55:49,112 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=17cfe364c2325fc5b7f2356e26a73a8e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:55:49,112 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c2a02331cabd6ae065e4a409f788b22c, ASSIGN 2023-02-11 19:55:49,113 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c2a02331cabd6ae065e4a409f788b22c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:55:49,262 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 19:55:49,264 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=17cfe364c2325fc5b7f2356e26a73a8e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:49,264 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349264"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145349264"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145349264"}]},"ts":"1676145349264"} 2023-02-11 19:55:49,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 17cfe364c2325fc5b7f2356e26a73a8e, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:49,267 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=c2a02331cabd6ae065e4a409f788b22c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:49,267 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349267"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145349267"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145349267"}]},"ts":"1676145349267"} 2023-02-11 19:55:49,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure c2a02331cabd6ae065e4a409f788b22c, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:49,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e. 2023-02-11 19:55:49,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 17cfe364c2325fc5b7f2356e26a73a8e, NAME => 'testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.', STARTKEY => '71', ENDKEY => ''} 2023-02-11 19:55:49,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:49,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:49,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,420 INFO [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,421 DEBUG [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/cf1 2023-02-11 19:55:49,421 DEBUG [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/cf1 2023-02-11 19:55:49,421 INFO [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 17cfe364c2325fc5b7f2356e26a73a8e columnFamilyName cf1 2023-02-11 19:55:49,422 INFO [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] regionserver.HStore(310): Store=17cfe364c2325fc5b7f2356e26a73a8e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:49,422 INFO [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,423 DEBUG [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/cf2 2023-02-11 19:55:49,423 DEBUG [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/cf2 2023-02-11 19:55:49,423 INFO [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 17cfe364c2325fc5b7f2356e26a73a8e columnFamilyName cf2 2023-02-11 19:55:49,424 INFO [StoreOpener-17cfe364c2325fc5b7f2356e26a73a8e-1] regionserver.HStore(310): Store=17cfe364c2325fc5b7f2356e26a73a8e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:49,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:49,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 17cfe364c2325fc5b7f2356e26a73a8e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67652868, jitterRate=0.0081062912940979}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:49,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 17cfe364c2325fc5b7f2356e26a73a8e: Running coprocessor pre-open hook at 1676145349419Writing region info on filesystem at 1676145349419Initializing all the Stores at 1676145349420 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145349420Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145349420Cleaning up temporary data from old regions at 1676145349425 (+5 ms)Cleaning up detritus from prior splits at 1676145349425Running coprocessor post-open hooks at 1676145349427 (+2 ms)Region opened successfully at 1676145349427 2023-02-11 19:55:49,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e., pid=133, masterSystemTime=1676145349416 2023-02-11 19:55:49,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e. 2023-02-11 19:55:49,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e. 2023-02-11 19:55:49,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c2a02331cabd6ae065e4a409f788b22c, NAME => 'testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.', STARTKEY => '', ENDKEY => '71'} 2023-02-11 19:55:49,429 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=17cfe364c2325fc5b7f2356e26a73a8e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:49,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:49,430 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349429"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145349429"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145349429"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145349429"}]},"ts":"1676145349429"} 2023-02-11 19:55:49,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:49,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,431 INFO [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,432 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1 2023-02-11 19:55:49,432 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1 2023-02-11 19:55:49,432 INFO [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c2a02331cabd6ae065e4a409f788b22c columnFamilyName cf1 2023-02-11 19:55:49,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-11 19:55:49,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 17cfe364c2325fc5b7f2356e26a73a8e, server=jenkins-hbase4.apache.org,42733,1676145241889 in 166 msec 2023-02-11 19:55:49,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=17cfe364c2325fc5b7f2356e26a73a8e, ASSIGN in 321 msec 2023-02-11 19:55:49,437 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-1f6cdd34396f4979a14aa180c4b7b903 2023-02-11 19:55:49,439 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-321a850bea16494e9e803a7804012bc8 2023-02-11 19:55:49,442 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-61bb9f0b4a32498c97375c67e648e21e 2023-02-11 19:55:49,444 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-72c50894d00c43be8e6a008ca54967e2 2023-02-11 19:55:49,447 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-a1673d2d88a24364a8dad077cb11d3c1 2023-02-11 19:55:49,449 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b1260c0766c04aceb93c9dda593578cd 2023-02-11 19:55:49,452 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-f56fcfc7f14e42ecb7336b25c5b816fc 2023-02-11 19:55:49,452 INFO [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(310): Store=c2a02331cabd6ae065e4a409f788b22c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:49,452 INFO [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,453 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2 2023-02-11 19:55:49,453 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2 2023-02-11 19:55:49,453 INFO [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c2a02331cabd6ae065e4a409f788b22c columnFamilyName cf2 2023-02-11 19:55:49,458 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-06b8968eea244fbcae14f24a70890d41 2023-02-11 19:55:49,462 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-0e20e087087a48648a61838965f7616b 2023-02-11 19:55:49,465 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-2233a757c6ab4a85b7d9e3d1e28d1f8c 2023-02-11 19:55:49,467 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-7c083124d811411fb635f731086c8e2e 2023-02-11 19:55:49,469 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-954149af44924a4abfcebc19bd47ef44 2023-02-11 19:55:49,472 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b746d6d0bac14cc18d77b58ec579fc6b 2023-02-11 19:55:49,474 DEBUG [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-ebb7b6a00a624326b178ce2af9a2765e 2023-02-11 19:55:49,475 INFO [StoreOpener-c2a02331cabd6ae065e4a409f788b22c-1] regionserver.HStore(310): Store=c2a02331cabd6ae065e4a409f788b22c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:49,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:49,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c2a02331cabd6ae065e4a409f788b22c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66868281, jitterRate=-0.0035849660634994507}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:49,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c2a02331cabd6ae065e4a409f788b22c: Running coprocessor pre-open hook at 1676145349430Writing region info on filesystem at 1676145349430Initializing all the Stores at 1676145349431 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145349431Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145349431Cleaning up temporary data from old regions at 1676145349476 (+45 ms)Cleaning up detritus from prior splits at 1676145349477 (+1 ms)Running coprocessor post-open hooks at 1676145349479 (+2 ms)Region opened successfully at 1676145349479 2023-02-11 19:55:49,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c., pid=134, masterSystemTime=1676145349416 2023-02-11 19:55:49,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:49,480 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:49,480 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:49,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:55:49,480 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 19:55:49,481 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 19:55:49,481 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): c2a02331cabd6ae065e4a409f788b22c/cf1 is initiating minor compaction (all files) 2023-02-11 19:55:49,481 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): c2a02331cabd6ae065e4a409f788b22c/cf2 is initiating minor compaction (all files) 2023-02-11 19:55:49,481 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c2a02331cabd6ae065e4a409f788b22c/cf1 in testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,481 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c2a02331cabd6ae065e4a409f788b22c/cf2 in testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,481 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-1f6cdd34396f4979a14aa180c4b7b903, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-61bb9f0b4a32498c97375c67e648e21e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b1260c0766c04aceb93c9dda593578cd, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-a1673d2d88a24364a8dad077cb11d3c1, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-f56fcfc7f14e42ecb7336b25c5b816fc, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-72c50894d00c43be8e6a008ca54967e2, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-321a850bea16494e9e803a7804012bc8] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/.tmp, totalSize=34.9 K 2023-02-11 19:55:49,481 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b746d6d0bac14cc18d77b58ec579fc6b, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-2233a757c6ab4a85b7d9e3d1e28d1f8c, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-06b8968eea244fbcae14f24a70890d41, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-ebb7b6a00a624326b178ce2af9a2765e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-7c083124d811411fb635f731086c8e2e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-954149af44924a4abfcebc19bd47ef44, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-0e20e087087a48648a61838965f7616b] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/.tmp, totalSize=34.9 K 2023-02-11 19:55:49,481 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-1f6cdd34396f4979a14aa180c4b7b903, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145344907 2023-02-11 19:55:49,481 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b746d6d0bac14cc18d77b58ec579fc6b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145344907 2023-02-11 19:55:49,482 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-61bb9f0b4a32498c97375c67e648e21e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145345485 2023-02-11 19:55:49,482 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-2233a757c6ab4a85b7d9e3d1e28d1f8c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145345485 2023-02-11 19:55:49,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,482 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b1260c0766c04aceb93c9dda593578cd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145346102 2023-02-11 19:55:49,482 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-06b8968eea244fbcae14f24a70890d41, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145346102 2023-02-11 19:55:49,482 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=c2a02331cabd6ae065e4a409f788b22c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:49,482 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-a1673d2d88a24364a8dad077cb11d3c1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145346690 2023-02-11 19:55:49,482 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-ebb7b6a00a624326b178ce2af9a2765e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145346690 2023-02-11 19:55:49,482 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349482"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145349482"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145349482"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145349482"}]},"ts":"1676145349482"} 2023-02-11 19:55:49,483 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-7c083124d811411fb635f731086c8e2e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145347267 2023-02-11 19:55:49,483 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-f56fcfc7f14e42ecb7336b25c5b816fc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145347267 2023-02-11 19:55:49,483 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-954149af44924a4abfcebc19bd47ef44, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145347433 2023-02-11 19:55:49,483 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-72c50894d00c43be8e6a008ca54967e2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145347433 2023-02-11 19:55:49,483 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-0e20e087087a48648a61838965f7616b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145347999 2023-02-11 19:55:49,484 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-321a850bea16494e9e803a7804012bc8, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145347999 2023-02-11 19:55:49,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-11 19:55:49,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure c2a02331cabd6ae065e4a409f788b22c, server=jenkins-hbase4.apache.org,42733,1676145241889 in 215 msec 2023-02-11 19:55:49,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-11 19:55:49,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c2a02331cabd6ae065e4a409f788b22c, ASSIGN in 375 msec 2023-02-11 19:55:49,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b355cf38157b492859da4977d8fc798e, daughterA=c2a02331cabd6ae065e4a409f788b22c, daughterB=17cfe364c2325fc5b7f2356e26a73a8e in 1.4830 sec 2023-02-11 19:55:49,503 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): c2a02331cabd6ae065e4a409f788b22c#cf1#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:49,503 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): c2a02331cabd6ae065e4a409f788b22c#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:49,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42733] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:45194 deadline: 1676145409509, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e. is not online on jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:49,524 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/.tmp/cf1/1423e71bed5d4fbc955d41ec2a410ffa as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/1423e71bed5d4fbc955d41ec2a410ffa 2023-02-11 19:55:49,524 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/.tmp/cf2/bc084740b8fa4d45a34a7ce064a20409 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/bc084740b8fa4d45a34a7ce064a20409 2023-02-11 19:55:49,528 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c2a02331cabd6ae065e4a409f788b22c/cf1 of c2a02331cabd6ae065e4a409f788b22c into 1423e71bed5d4fbc955d41ec2a410ffa(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:49,528 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c2a02331cabd6ae065e4a409f788b22c/cf2 of c2a02331cabd6ae065e4a409f788b22c into bc084740b8fa4d45a34a7ce064a20409(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:49,528 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c2a02331cabd6ae065e4a409f788b22c: 2023-02-11 19:55:49,528 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c2a02331cabd6ae065e4a409f788b22c: 2023-02-11 19:55:49,528 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c., storeName=c2a02331cabd6ae065e4a409f788b22c/cf1, priority=9, startTime=1676145349479; duration=0sec 2023-02-11 19:55:49,528 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c., storeName=c2a02331cabd6ae065e4a409f788b22c/cf2, priority=9, startTime=1676145349480; duration=0sec 2023-02-11 19:55:49,528 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:49,529 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:49,625 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-11 19:55:49,629 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:49,630 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,630 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 19:55:49,633 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145349633"}]},"ts":"1676145349633"} 2023-02-11 19:55:49,634 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-11 19:55:49,635 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-11 19:55:49,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c2a02331cabd6ae065e4a409f788b22c, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=17cfe364c2325fc5b7f2356e26a73a8e, UNASSIGN}] 2023-02-11 19:55:49,636 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=17cfe364c2325fc5b7f2356e26a73a8e, UNASSIGN 2023-02-11 19:55:49,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=17cfe364c2325fc5b7f2356e26a73a8e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:49,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145349636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145349636"}]},"ts":"1676145349636"} 2023-02-11 19:55:49,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 17cfe364c2325fc5b7f2356e26a73a8e, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:49,639 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c2a02331cabd6ae065e4a409f788b22c, UNASSIGN 2023-02-11 19:55:49,639 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=c2a02331cabd6ae065e4a409f788b22c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:49,639 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349639"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145349639"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145349639"}]},"ts":"1676145349639"} 2023-02-11 19:55:49,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure c2a02331cabd6ae065e4a409f788b22c, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:49,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 19:55:49,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c2a02331cabd6ae065e4a409f788b22c, disabling compactions & flushes 2023-02-11 19:55:49,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. after waiting 0 ms 2023-02-11 19:55:49,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,790 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-1f6cdd34396f4979a14aa180c4b7b903, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-61bb9f0b4a32498c97375c67e648e21e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b1260c0766c04aceb93c9dda593578cd, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-a1673d2d88a24364a8dad077cb11d3c1, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-f56fcfc7f14e42ecb7336b25c5b816fc, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-72c50894d00c43be8e6a008ca54967e2, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-321a850bea16494e9e803a7804012bc8] to archive 2023-02-11 19:55:49,791 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:49,792 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-1f6cdd34396f4979a14aa180c4b7b903 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-1f6cdd34396f4979a14aa180c4b7b903 2023-02-11 19:55:49,793 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-61bb9f0b4a32498c97375c67e648e21e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-61bb9f0b4a32498c97375c67e648e21e 2023-02-11 19:55:49,794 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b1260c0766c04aceb93c9dda593578cd to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b1260c0766c04aceb93c9dda593578cd 2023-02-11 19:55:49,795 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-a1673d2d88a24364a8dad077cb11d3c1 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-a1673d2d88a24364a8dad077cb11d3c1 2023-02-11 19:55:49,795 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-f56fcfc7f14e42ecb7336b25c5b816fc to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-f56fcfc7f14e42ecb7336b25c5b816fc 2023-02-11 19:55:49,796 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-72c50894d00c43be8e6a008ca54967e2 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-72c50894d00c43be8e6a008ca54967e2 2023-02-11 19:55:49,797 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-321a850bea16494e9e803a7804012bc8 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-321a850bea16494e9e803a7804012bc8 2023-02-11 19:55:49,799 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b746d6d0bac14cc18d77b58ec579fc6b, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-2233a757c6ab4a85b7d9e3d1e28d1f8c, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-06b8968eea244fbcae14f24a70890d41, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-ebb7b6a00a624326b178ce2af9a2765e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-7c083124d811411fb635f731086c8e2e, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-954149af44924a4abfcebc19bd47ef44, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-0e20e087087a48648a61838965f7616b] to archive 2023-02-11 19:55:49,799 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:49,800 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b746d6d0bac14cc18d77b58ec579fc6b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-b746d6d0bac14cc18d77b58ec579fc6b 2023-02-11 19:55:49,801 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-2233a757c6ab4a85b7d9e3d1e28d1f8c to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-2233a757c6ab4a85b7d9e3d1e28d1f8c 2023-02-11 19:55:49,802 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-06b8968eea244fbcae14f24a70890d41 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-06b8968eea244fbcae14f24a70890d41 2023-02-11 19:55:49,803 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-ebb7b6a00a624326b178ce2af9a2765e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-ebb7b6a00a624326b178ce2af9a2765e 2023-02-11 19:55:49,804 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-7c083124d811411fb635f731086c8e2e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-7c083124d811411fb635f731086c8e2e 2023-02-11 19:55:49,805 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-954149af44924a4abfcebc19bd47ef44 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-954149af44924a4abfcebc19bd47ef44 2023-02-11 19:55:49,805 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-0e20e087087a48648a61838965f7616b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/testSplitTableRegionEmptyDaughter=b355cf38157b492859da4977d8fc798e-0e20e087087a48648a61838965f7616b 2023-02-11 19:55:49,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:55:49,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:49,809 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c. 2023-02-11 19:55:49,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c2a02331cabd6ae065e4a409f788b22c: Waiting for close lock at 1676145349789Running coprocessor pre-close hooks at 1676145349789Disabling compacts and flushes for region at 1676145349789Disabling writes for close at 1676145349789Writing region close event to WAL at 1676145349806 (+17 ms)Running coprocessor post-close hooks at 1676145349809 (+3 ms)Closed at 1676145349809 2023-02-11 19:55:49,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 17cfe364c2325fc5b7f2356e26a73a8e, disabling compactions & flushes 2023-02-11 19:55:49,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e. 2023-02-11 19:55:49,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e. 2023-02-11 19:55:49,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e. after waiting 0 ms 2023-02-11 19:55:49,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e. 2023-02-11 19:55:49,811 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=c2a02331cabd6ae065e4a409f788b22c, regionState=CLOSED 2023-02-11 19:55:49,811 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349811"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145349811"}]},"ts":"1676145349811"} 2023-02-11 19:55:49,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-11 19:55:49,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure c2a02331cabd6ae065e4a409f788b22c, server=jenkins-hbase4.apache.org,42733,1676145241889 in 172 msec 2023-02-11 19:55:49,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-11 19:55:49,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:49,814 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e. 2023-02-11 19:55:49,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 17cfe364c2325fc5b7f2356e26a73a8e: Waiting for close lock at 1676145349810Running coprocessor pre-close hooks at 1676145349810Disabling compacts and flushes for region at 1676145349810Disabling writes for close at 1676145349811 (+1 ms)Writing region close event to WAL at 1676145349811Running coprocessor post-close hooks at 1676145349814 (+3 ms)Closed at 1676145349814 2023-02-11 19:55:49,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c2a02331cabd6ae065e4a409f788b22c, UNASSIGN in 179 msec 2023-02-11 19:55:49,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=17cfe364c2325fc5b7f2356e26a73a8e, regionState=CLOSED 2023-02-11 19:55:49,815 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676145349815"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145349815"}]},"ts":"1676145349815"} 2023-02-11 19:55:49,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-11 19:55:49,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 17cfe364c2325fc5b7f2356e26a73a8e, server=jenkins-hbase4.apache.org,42733,1676145241889 in 179 msec 2023-02-11 19:55:49,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-11 19:55:49,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=17cfe364c2325fc5b7f2356e26a73a8e, UNASSIGN in 182 msec 2023-02-11 19:55:49,819 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145349819"}]},"ts":"1676145349819"} 2023-02-11 19:55:49,820 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-11 19:55:49,822 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-11 19:55:49,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 192 msec 2023-02-11 19:55:49,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 19:55:49,935 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-11 19:55:49,935 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,938 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,939 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-11 19:55:49,942 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:49,942 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,942 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,944 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/recovered.edits] 2023-02-11 19:55:49,944 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/recovered.edits] 2023-02-11 19:55:49,944 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/recovered.edits] 2023-02-11 19:55:49,948 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/1423e71bed5d4fbc955d41ec2a410ffa to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf1/1423e71bed5d4fbc955d41ec2a410ffa 2023-02-11 19:55:49,948 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/1f6cdd34396f4979a14aa180c4b7b903 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/1f6cdd34396f4979a14aa180c4b7b903 2023-02-11 19:55:49,950 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/321a850bea16494e9e803a7804012bc8 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/321a850bea16494e9e803a7804012bc8 2023-02-11 19:55:49,950 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/recovered.edits/88.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e/recovered.edits/88.seqid 2023-02-11 19:55:49,950 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/bc084740b8fa4d45a34a7ce064a20409 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/cf2/bc084740b8fa4d45a34a7ce064a20409 2023-02-11 19:55:49,951 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/61bb9f0b4a32498c97375c67e648e21e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/61bb9f0b4a32498c97375c67e648e21e 2023-02-11 19:55:49,951 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/17cfe364c2325fc5b7f2356e26a73a8e 2023-02-11 19:55:49,952 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/72c50894d00c43be8e6a008ca54967e2 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/72c50894d00c43be8e6a008ca54967e2 2023-02-11 19:55:49,953 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c/recovered.edits/90.seqid 2023-02-11 19:55:49,953 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/a1673d2d88a24364a8dad077cb11d3c1 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/a1673d2d88a24364a8dad077cb11d3c1 2023-02-11 19:55:49,953 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/c2a02331cabd6ae065e4a409f788b22c 2023-02-11 19:55:49,954 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/b1260c0766c04aceb93c9dda593578cd to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/b1260c0766c04aceb93c9dda593578cd 2023-02-11 19:55:49,955 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/f56fcfc7f14e42ecb7336b25c5b816fc to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf1/f56fcfc7f14e42ecb7336b25c5b816fc 2023-02-11 19:55:49,957 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/06b8968eea244fbcae14f24a70890d41 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/06b8968eea244fbcae14f24a70890d41 2023-02-11 19:55:49,958 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/0e20e087087a48648a61838965f7616b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/0e20e087087a48648a61838965f7616b 2023-02-11 19:55:49,959 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/2233a757c6ab4a85b7d9e3d1e28d1f8c 2023-02-11 19:55:49,959 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/7c083124d811411fb635f731086c8e2e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/7c083124d811411fb635f731086c8e2e 2023-02-11 19:55:49,960 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/954149af44924a4abfcebc19bd47ef44 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/954149af44924a4abfcebc19bd47ef44 2023-02-11 19:55:49,961 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/b746d6d0bac14cc18d77b58ec579fc6b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/b746d6d0bac14cc18d77b58ec579fc6b 2023-02-11 19:55:49,962 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/ebb7b6a00a624326b178ce2af9a2765e to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/cf2/ebb7b6a00a624326b178ce2af9a2765e 2023-02-11 19:55:49,963 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/recovered.edits/85.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e/recovered.edits/85.seqid 2023-02-11 19:55:49,964 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegionEmptyDaughter/b355cf38157b492859da4977d8fc798e 2023-02-11 19:55:49,964 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-11 19:55:49,965 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,967 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-11 19:55:49,969 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-11 19:55:49,969 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,969 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-11 19:55:49,970 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145349969"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:49,970 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145349969"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:49,970 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145349969"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:49,971 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 19:55:49,971 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b355cf38157b492859da4977d8fc798e, NAME => 'testSplitTableRegionEmptyDaughter,,1676145344403.b355cf38157b492859da4977d8fc798e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c2a02331cabd6ae065e4a409f788b22c, NAME => 'testSplitTableRegionEmptyDaughter,,1676145348005.c2a02331cabd6ae065e4a409f788b22c.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 17cfe364c2325fc5b7f2356e26a73a8e, NAME => 'testSplitTableRegionEmptyDaughter,71,1676145348005.17cfe364c2325fc5b7f2356e26a73a8e.', STARTKEY => '71', ENDKEY => ''}] 2023-02-11 19:55:49,971 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-11 19:55:49,971 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145349971"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:49,972 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-11 19:55:49,974 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 19:55:49,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 38 msec 2023-02-11 19:55:50,017 ERROR [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 1d17abb3-77b3-4fd1-a223-1cc76825d5c0) service to localhost/127.0.0.1:42039 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:55:50,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-11 19:55:50,040 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-11 19:55:50,053 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=373 (was 374), OpenFileDescriptor=841 (was 841), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=149 (was 135) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3369 (was 3421) 2023-02-11 19:55:50,066 INFO [Listener at localhost/38771] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=373, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=149, ProcessCount=168, AvailableMemoryMB=3368 2023-02-11 19:55:50,067 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:50,069 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 19:55:50,069 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 19:55:50,070 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-11 19:55:50,071 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 19:55:50,072 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 19:55:50,073 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,073 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998 empty. 2023-02-11 19:55:50,074 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,074 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-11 19:55:50,082 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-11 19:55:50,083 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => b9fef38429861932e92e13f6b792d998, NAME => 'testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp 2023-02-11 19:55:50,491 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:50,491 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing b9fef38429861932e92e13f6b792d998, disabling compactions & flushes 2023-02-11 19:55:50,492 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:50,492 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:50,492 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. after waiting 0 ms 2023-02-11 19:55:50,492 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:50,492 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:50,492 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for b9fef38429861932e92e13f6b792d998: Waiting for close lock at 1676145350491Disabling compacts and flushes for region at 1676145350491Disabling writes for close at 1676145350492 (+1 ms)Writing region close event to WAL at 1676145350492Closed at 1676145350492 2023-02-11 19:55:50,494 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 19:55:50,494 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145350494"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145350494"}]},"ts":"1676145350494"} 2023-02-11 19:55:50,495 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 19:55:50,496 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 19:55:50,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145350496"}]},"ts":"1676145350496"} 2023-02-11 19:55:50,497 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-11 19:55:50,505 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 19:55:50,506 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 19:55:50,506 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 19:55:50,506 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 19:55:50,506 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 19:55:50,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=b9fef38429861932e92e13f6b792d998, ASSIGN}] 2023-02-11 19:55:50,507 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=b9fef38429861932e92e13f6b792d998, ASSIGN 2023-02-11 19:55:50,508 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=b9fef38429861932e92e13f6b792d998, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:55:50,658 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 19:55:50,659 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=b9fef38429861932e92e13f6b792d998, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:50,659 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145350659"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145350659"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145350659"}]},"ts":"1676145350659"} 2023-02-11 19:55:50,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure b9fef38429861932e92e13f6b792d998, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:50,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:50,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b9fef38429861932e92e13f6b792d998, NAME => 'testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.', STARTKEY => '', ENDKEY => ''} 2023-02-11 19:55:50,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:50,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:50,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,815 INFO [StoreOpener-b9fef38429861932e92e13f6b792d998-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,816 DEBUG [StoreOpener-b9fef38429861932e92e13f6b792d998-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1 2023-02-11 19:55:50,817 DEBUG [StoreOpener-b9fef38429861932e92e13f6b792d998-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1 2023-02-11 19:55:50,817 INFO [StoreOpener-b9fef38429861932e92e13f6b792d998-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9fef38429861932e92e13f6b792d998 columnFamilyName cf1 2023-02-11 19:55:50,817 INFO [StoreOpener-b9fef38429861932e92e13f6b792d998-1] regionserver.HStore(310): Store=b9fef38429861932e92e13f6b792d998/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:50,817 INFO [StoreOpener-b9fef38429861932e92e13f6b792d998-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,818 DEBUG [StoreOpener-b9fef38429861932e92e13f6b792d998-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2 2023-02-11 19:55:50,818 DEBUG [StoreOpener-b9fef38429861932e92e13f6b792d998-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2 2023-02-11 19:55:50,818 INFO [StoreOpener-b9fef38429861932e92e13f6b792d998-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9fef38429861932e92e13f6b792d998 columnFamilyName cf2 2023-02-11 19:55:50,819 INFO [StoreOpener-b9fef38429861932e92e13f6b792d998-1] regionserver.HStore(310): Store=b9fef38429861932e92e13f6b792d998/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:50,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:50,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:50,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 19:55:50,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b9fef38429861932e92e13f6b792d998; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62232948, jitterRate=-0.07265681028366089}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:50,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b9fef38429861932e92e13f6b792d998: Running coprocessor pre-open hook at 1676145350815Writing region info on filesystem at 1676145350815Initializing all the Stores at 1676145350815Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145350815Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145350815Cleaning up temporary data from old regions at 1676145350820 (+5 ms)Cleaning up detritus from prior splits at 1676145350820Running coprocessor post-open hooks at 1676145350823 (+3 ms)Region opened successfully at 1676145350823 2023-02-11 19:55:50,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998., pid=143, masterSystemTime=1676145350812 2023-02-11 19:55:50,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:50,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:50,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=b9fef38429861932e92e13f6b792d998, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:50,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145350825"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145350825"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145350825"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145350825"}]},"ts":"1676145350825"} 2023-02-11 19:55:50,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-11 19:55:50,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure b9fef38429861932e92e13f6b792d998, server=jenkins-hbase4.apache.org,42733,1676145241889 in 166 msec 2023-02-11 19:55:50,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-11 19:55:50,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=b9fef38429861932e92e13f6b792d998, ASSIGN in 322 msec 2023-02-11 19:55:50,829 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 19:55:50,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145350829"}]},"ts":"1676145350829"} 2023-02-11 19:55:50,830 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-11 19:55:50,832 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 19:55:50,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 762 msec 2023-02-11 19:55:51,077 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:51,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 19:55:51,078 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 19:55:51,078 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:51,078 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 19:55:51,078 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 19:55:51,079 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,079 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,080 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,080 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,080 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,080 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,080 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,080 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:51,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:51,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:51,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:51,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:51,083 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:51,083 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. started... 2023-02-11 19:55:51,083 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9fef38429861932e92e13f6b792d998 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 19:55:51,084 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,088 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,088 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:51,088 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,089 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,089 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:51,221 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-11 19:55:51,492 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/eea19316baea406491729f2686e1656b 2023-02-11 19:55:51,502 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/1b9b9e1f27914d93a33823ff7167592f 2023-02-11 19:55:51,505 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/eea19316baea406491729f2686e1656b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b 2023-02-11 19:55:51,508 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:51,509 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/1b9b9e1f27914d93a33823ff7167592f as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f 2023-02-11 19:55:51,511 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:51,512 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b9fef38429861932e92e13f6b792d998 in 429ms, sequenceid=6, compaction requested=false 2023-02-11 19:55:51,513 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9fef38429861932e92e13f6b792d998: 2023-02-11 19:55:51,513 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:51,513 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:51,513 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:51,513 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,513 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:51,513 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:51,514 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,514 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:51,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:51,515 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,515 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:51,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:51,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:51,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:51,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:51,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 19:55:51,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16d119c3[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:51,516 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 19:55:51,516 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,517 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,517 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,517 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,518 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:51,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:51,518 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:51,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:51,518 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:51,519 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:51,519 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:51,519 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:51,523 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,523 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:51,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:51,523 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:51,523 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:51,524 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:51,524 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:51,524 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:51,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:51,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:51,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:51,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:51,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:51,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:51,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 19:55:51,526 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 19:55:51,526 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:51,526 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:51,526 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 19:55:51,526 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:51,527 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,527 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:51,527 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,527 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,528 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:51,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,528 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:51,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:51,527 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:51,528 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:51,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:51,528 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:51,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,529 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:51,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:51,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,530 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:51,531 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:51,531 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,533 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,533 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,533 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,533 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:51,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,533 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,533 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,533 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 19:55:51,533 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:51,533 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:51,533 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:51,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,533 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:51,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:51,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,534 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,634 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:51,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:51,643 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:51,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:51,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 19:55:51,645 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 19:55:51,645 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:51,645 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 19:55:51,645 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 19:55:51,645 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,645 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,646 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,646 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,646 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,646 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:51,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:51,646 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:51,647 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,647 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:51,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:51,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:51,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:51,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:51,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:51,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:51,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:51,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:51,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:51,648 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:51,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:51,649 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:51,649 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:51,649 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:51,649 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. started... 2023-02-11 19:55:51,649 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:51,649 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:51,649 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:51,649 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9fef38429861932e92e13f6b792d998 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:51,651 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,651 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,651 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:51,652 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,652 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:51,652 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:51,658 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/a3738cce04ca43dca5f03e7583d1c259 2023-02-11 19:55:52,070 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/a5ab8f5bca004d23ab463361642db3d8 2023-02-11 19:55:52,074 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/a3738cce04ca43dca5f03e7583d1c259 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259 2023-02-11 19:55:52,078 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:55:52,078 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/a5ab8f5bca004d23ab463361642db3d8 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8 2023-02-11 19:55:52,082 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 19:55:52,083 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b9fef38429861932e92e13f6b792d998 in 434ms, sequenceid=19, compaction requested=false 2023-02-11 19:55:52,083 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9fef38429861932e92e13f6b792d998: 2023-02-11 19:55:52,083 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,083 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:52,083 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:52,083 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,083 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,083 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,085 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,085 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,085 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,085 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 19:55:52,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c951716[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:52,088 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 19:55:52,088 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,089 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,089 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,089 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,089 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,089 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,089 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,089 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:52,089 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,092 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:52,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,094 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,094 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,094 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:52,096 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,096 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:52,098 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,098 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,098 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,098 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,098 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,098 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,098 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,099 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,099 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,099 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:52,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 19:55:52,102 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 19:55:52,102 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:52,102 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:52,102 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 19:55:52,102 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:52,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,104 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,104 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,105 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,105 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,107 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,108 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,108 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,111 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,111 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,111 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 19:55:52,112 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:52,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,112 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:52,213 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:52,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:52,223 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:52,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:52,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 19:55:52,224 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 19:55:52,224 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,224 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 19:55:52,224 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 19:55:52,225 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,225 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,226 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,226 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,226 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,226 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,226 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,226 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:52,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,227 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:52,228 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:52,228 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,228 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. started... 2023-02-11 19:55:52,229 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9fef38429861932e92e13f6b792d998 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:52,230 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,235 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,235 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,235 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,236 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,236 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,238 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/a32d7b2977af4a5ea32db70cd3a1c568 2023-02-11 19:55:52,253 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/c2b4237dd67e4095b293a62013077a37 2023-02-11 19:55:52,257 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/a32d7b2977af4a5ea32db70cd3a1c568 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568 2023-02-11 19:55:52,261 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:55:52,262 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/c2b4237dd67e4095b293a62013077a37 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37 2023-02-11 19:55:52,266 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 19:55:52,266 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b9fef38429861932e92e13f6b792d998 in 37ms, sequenceid=32, compaction requested=true 2023-02-11 19:55:52,267 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9fef38429861932e92e13f6b792d998: 2023-02-11 19:55:52,267 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,267 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:52,267 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:52,267 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,267 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,267 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,268 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,268 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,269 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,269 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 19:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@743ffca0[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:52,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 19:55:52,271 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,279 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:52,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:52,279 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:52,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,281 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,281 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,281 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,281 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,281 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,281 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,281 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,282 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:52,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 19:55:52,285 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 19:55:52,285 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:52,285 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:52,285 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 19:55:52,285 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:52,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,287 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,288 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,288 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,289 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,290 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,291 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,291 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,294 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,295 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 19:55:52,295 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:52,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,295 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:52,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,295 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,296 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,296 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,396 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:52,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:52,406 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:52,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:52,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 19:55:52,407 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 19:55:52,407 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,407 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 19:55:52,407 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 19:55:52,408 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,408 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,409 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,409 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,409 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,409 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,409 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,409 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:52,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,411 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,411 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,411 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:52,411 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:52,412 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,412 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. started... 2023-02-11 19:55:52,412 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9fef38429861932e92e13f6b792d998 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:52,412 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,412 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,413 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,413 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,413 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,413 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,422 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/ae7da5799ee44558b2f56ded9ca967d6 2023-02-11 19:55:52,435 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/5f7c838085b94e48b651b6bb13b78d25 2023-02-11 19:55:52,443 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/ae7da5799ee44558b2f56ded9ca967d6 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6 2023-02-11 19:55:52,446 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:55:52,447 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/5f7c838085b94e48b651b6bb13b78d25 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25 2023-02-11 19:55:52,449 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 19:55:52,450 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b9fef38429861932e92e13f6b792d998 in 38ms, sequenceid=45, compaction requested=true 2023-02-11 19:55:52,450 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9fef38429861932e92e13f6b792d998: 2023-02-11 19:55:52,450 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,450 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:52,450 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:52,450 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,450 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,450 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,452 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,452 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,452 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,452 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 19:55:52,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4523149[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:52,454 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 19:55:52,455 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,457 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,458 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:52,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,458 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:52,458 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:52,458 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,458 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,458 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:52,459 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,459 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,459 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,459 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,459 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,460 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,460 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,460 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,460 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,460 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:52,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 19:55:52,463 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 19:55:52,463 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:52,463 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:52,463 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 19:55:52,463 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:52,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,466 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,467 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,467 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,468 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,470 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,470 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,470 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,476 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 19:55:52,476 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,477 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:52,477 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:52,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,476 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,476 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,477 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,477 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,477 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,477 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,477 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,477 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,478 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,577 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:52,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:52,587 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:52,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:52,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 19:55:52,588 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 19:55:52,588 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,588 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 19:55:52,588 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 19:55:52,588 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,588 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,589 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,589 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,590 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,590 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,592 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,592 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,591 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,592 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,592 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,592 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:52,592 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:52,592 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,592 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. started... 2023-02-11 19:55:52,592 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9fef38429861932e92e13f6b792d998 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:52,593 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,593 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,593 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,593 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,593 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,594 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,600 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/a781238f991b460494baf8c420770f73 2023-02-11 19:55:52,609 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/4c37e969e3d74ee7999315a7d0532a20 2023-02-11 19:55:52,612 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/a781238f991b460494baf8c420770f73 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73 2023-02-11 19:55:52,614 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:55:52,615 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/4c37e969e3d74ee7999315a7d0532a20 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20 2023-02-11 19:55:52,618 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 19:55:52,618 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b9fef38429861932e92e13f6b792d998 in 26ms, sequenceid=58, compaction requested=true 2023-02-11 19:55:52,618 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9fef38429861932e92e13f6b792d998: 2023-02-11 19:55:52,618 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,618 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:52,618 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:52,619 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,619 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,619 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,620 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 19:55:52,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ad33f01[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:52,622 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 19:55:52,622 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,624 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,624 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:52,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,624 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,624 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,624 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,625 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:52,625 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,625 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,625 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,625 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:52,626 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,626 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,626 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,626 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,626 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,626 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,626 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,626 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,627 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,627 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:52,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 19:55:52,629 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 19:55:52,629 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:52,629 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:52,629 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 19:55:52,629 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:52,630 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,630 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,630 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,631 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,630 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,630 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,631 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,631 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,632 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,632 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,634 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,634 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,634 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,637 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 19:55:52,638 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,637 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,637 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,638 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,738 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:52,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:52,749 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 19:55:52,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 19:55:52,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 19:55:52,750 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 19:55:52,750 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,750 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 19:55:52,751 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 19:55:52,751 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,751 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,752 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,752 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,752 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,752 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,752 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,753 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 19:55:52,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,753 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 19:55:52,754 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,754 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,754 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,754 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,754 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43747,1676145241852' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 19:55:52,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 19:55:52,754 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,754 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 19:55:52,754 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 19:55:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 19:55:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 19:55:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 19:55:52,755 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,755 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32837,1676145241806' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,755 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. started... 2023-02-11 19:55:52,755 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9fef38429861932e92e13f6b792d998 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 19:55:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,756 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,757 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,757 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,769 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/0bb6af9ef2b5464ba2eada69a97e6d67 2023-02-11 19:55:52,780 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/c11361d8e1034bcabf76a719b5fdc698 2023-02-11 19:55:52,783 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/0bb6af9ef2b5464ba2eada69a97e6d67 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67 2023-02-11 19:55:52,786 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:55:52,786 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/c11361d8e1034bcabf76a719b5fdc698 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698 2023-02-11 19:55:52,789 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 19:55:52,790 INFO [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b9fef38429861932e92e13f6b792d998 in 35ms, sequenceid=71, compaction requested=true 2023-02-11 19:55:52,790 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9fef38429861932e92e13f6b792d998: 2023-02-11 19:55:52,790 DEBUG [rs(jenkins-hbase4.apache.org,42733,1676145241889)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:52,790 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 19:55:52,790 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 19:55:52,790 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,790 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 19:55:52,790 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 19:55:52,793 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,793 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 19:55:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42733,1676145241889' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 19:55:52,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e8ab6ea[Count = 0] remaining members to acquire global barrier 2023-02-11 19:55:52,795 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 19:55:52,795 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,796 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,796 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 19:55:52,796 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43747,1676145241852' in zk 2023-02-11 19:55:52,796 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,796 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,796 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,32837,1676145241806' in zk 2023-02-11 19:55:52,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,797 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 19:55:52,797 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 19:55:52,797 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42733,1676145241889' in zk 2023-02-11 19:55:52,800 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,800 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,800 DEBUG [member: 'jenkins-hbase4.apache.org,43747,1676145241852' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,801 DEBUG [member: 'jenkins-hbase4.apache.org,32837,1676145241806' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,802 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 19:55:52,802 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,802 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,802 DEBUG [member: 'jenkins-hbase4.apache.org,42733,1676145241889' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 19:55:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,42733,1676145241889': 2023-02-11 19:55:52,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42733,1676145241889' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 19:55:52,805 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 19:55:52,805 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 19:55:52,805 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 19:55:52,805 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 19:55:52,805 INFO [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 19:55:52,807 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 19:55:52,807 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,807 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 19:55:52,807 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 19:55:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,808 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,809 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 19:55:52,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,810 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 19:55:52,810 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 19:55:52,810 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,813 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 19:55:52,813 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,813 DEBUG [(jenkins-hbase4.apache.org,37191,1676145240670)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 19:55:52,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 19:55:52,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 19:55:52,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 19:55:52,813 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,814 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 19:55:52,814 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 19:55:52,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 19:55:52,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 19:55:52,814 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:52,814 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,814 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,814 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:52,814 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 19:55:52,814 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 19:55:52,914 DEBUG [Listener at localhost/38771] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 19:55:52,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 19:55:52,921 DEBUG [Listener at localhost/38771] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,922 DEBUG [Listener at localhost/38771] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=b9fef38429861932e92e13f6b792d998, daughterA=29bf5a2ffc186ac6925edb613a6518ba, daughterB=59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:52,923 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=b9fef38429861932e92e13f6b792d998, daughterA=29bf5a2ffc186ac6925edb613a6518ba, daughterB=59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:52,923 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=b9fef38429861932e92e13f6b792d998, daughterA=29bf5a2ffc186ac6925edb613a6518ba, daughterB=59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:52,923 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=b9fef38429861932e92e13f6b792d998, daughterA=29bf5a2ffc186ac6925edb613a6518ba, daughterB=59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:52,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=b9fef38429861932e92e13f6b792d998, UNASSIGN}] 2023-02-11 19:55:52,925 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=b9fef38429861932e92e13f6b792d998, UNASSIGN 2023-02-11 19:55:52,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=b9fef38429861932e92e13f6b792d998, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:52,926 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145352926"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145352926"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145352926"}]},"ts":"1676145352926"} 2023-02-11 19:55:52,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure b9fef38429861932e92e13f6b792d998, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:53,069 ERROR [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 2f25de57-daa1-42aa-aa7e-60a3bc1a94d0) service to localhost/127.0.0.1:42039 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 19:55:53,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b9fef38429861932e92e13f6b792d998, disabling compactions & flushes 2023-02-11 19:55:53,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:53,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:53,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. after waiting 0 ms 2023-02-11 19:55:53,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:53,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing b9fef38429861932e92e13f6b792d998 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 19:55:53,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/837659bd108742bb9977478deedabbdb 2023-02-11 19:55:53,104 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/22f30b5e76984cad933a0ec50fc8d878 2023-02-11 19:55:53,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf1/837659bd108742bb9977478deedabbdb as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb 2023-02-11 19:55:53,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:55:53,111 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/.tmp/cf2/22f30b5e76984cad933a0ec50fc8d878 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878 2023-02-11 19:55:53,114 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 19:55:53,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b9fef38429861932e92e13f6b792d998 in 36ms, sequenceid=82, compaction requested=true 2023-02-11 19:55:53,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 19:55:53,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:53,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. 2023-02-11 19:55:53,123 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b9fef38429861932e92e13f6b792d998: Waiting for close lock at 1676145353079Running coprocessor pre-close hooks at 1676145353079Disabling compacts and flushes for region at 1676145353079Disabling writes for close at 1676145353079Obtaining lock to block concurrent updates at 1676145353079Preparing flush snapshotting stores in b9fef38429861932e92e13f6b792d998 at 1676145353079Finished memstore snapshotting testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676145353079Flushing stores of testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. at 1676145353080 (+1 ms)Flushing b9fef38429861932e92e13f6b792d998/cf1: creating writer at 1676145353081 (+1 ms)Flushing b9fef38429861932e92e13f6b792d998/cf1: appending metadata at 1676145353082 (+1 ms)Flushing b9fef38429861932e92e13f6b792d998/cf1: closing flushed file at 1676145353082Flushing b9fef38429861932e92e13f6b792d998/cf2: creating writer at 1676145353092 (+10 ms)Flushing b9fef38429861932e92e13f6b792d998/cf2: appending metadata at 1676145353093 (+1 ms)Flushing b9fef38429861932e92e13f6b792d998/cf2: closing flushed file at 1676145353093Flushing b9fef38429861932e92e13f6b792d998/cf1: reopening flushed file at 1676145353108 (+15 ms)Flushing b9fef38429861932e92e13f6b792d998/cf2: reopening flushed file at 1676145353112 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b9fef38429861932e92e13f6b792d998 in 36ms, sequenceid=82, compaction requested=true at 1676145353115 (+3 ms)Writing region close event to WAL at 1676145353120 (+5 ms)Running coprocessor post-close hooks at 1676145353122 (+2 ms)Closed at 1676145353122 2023-02-11 19:55:53,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,124 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=b9fef38429861932e92e13f6b792d998, regionState=CLOSED 2023-02-11 19:55:53,124 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145353124"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145353124"}]},"ts":"1676145353124"} 2023-02-11 19:55:53,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-11 19:55:53,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure b9fef38429861932e92e13f6b792d998, server=jenkins-hbase4.apache.org,42733,1676145241889 in 198 msec 2023-02-11 19:55:53,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-11 19:55:53,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=b9fef38429861932e92e13f6b792d998, UNASSIGN in 202 msec 2023-02-11 19:55:53,137 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=b9fef38429861932e92e13f6b792d998, threads=14 2023-02-11 19:55:53,138 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,138 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,138 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,138 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,144 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,144 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,144 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,144 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,144 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,145 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,145 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,145 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,145 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,145 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,280 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,282 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,296 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,308 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,320 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,320 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,331 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,337 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,338 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,338 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,696 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 19:55:53,698 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,722 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:53,739 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:54,011 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73 for region: b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:54,011 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region b9fef38429861932e92e13f6b792d998 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 19:55:54,443 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:55:54,444 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 19:55:54,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676145354445"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676145354445"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676145354445"}]},"ts":"1676145354445"} 2023-02-11 19:55:54,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145354445"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145354445"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145354445"}]},"ts":"1676145354445"} 2023-02-11 19:55:54,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145354445"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145354445"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145354445"}]},"ts":"1676145354445"} 2023-02-11 19:55:54,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=29bf5a2ffc186ac6925edb613a6518ba, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=59d5d182806612f0afda82c67a7a2773, ASSIGN}] 2023-02-11 19:55:54,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=59d5d182806612f0afda82c67a7a2773, ASSIGN 2023-02-11 19:55:54,449 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=59d5d182806612f0afda82c67a7a2773, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:55:54,449 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=29bf5a2ffc186ac6925edb613a6518ba, ASSIGN 2023-02-11 19:55:54,449 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=29bf5a2ffc186ac6925edb613a6518ba, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42733,1676145241889; forceNewPlan=false, retain=false 2023-02-11 19:55:54,599 INFO [jenkins-hbase4:37191] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 19:55:54,600 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=29bf5a2ffc186ac6925edb613a6518ba, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:54,601 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145354600"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145354600"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145354600"}]},"ts":"1676145354600"} 2023-02-11 19:55:54,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 29bf5a2ffc186ac6925edb613a6518ba, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:54,604 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=59d5d182806612f0afda82c67a7a2773, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:54,604 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145354604"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145354604"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145354604"}]},"ts":"1676145354604"} 2023-02-11 19:55:54,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 59d5d182806612f0afda82c67a7a2773, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:54,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:54,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 59d5d182806612f0afda82c67a7a2773, NAME => 'testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 19:55:54,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:54,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:54,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:54,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:54,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:54,757 INFO [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:54,758 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1 2023-02-11 19:55:54,758 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1 2023-02-11 19:55:54,758 INFO [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 59d5d182806612f0afda82c67a7a2773 columnFamilyName cf1 2023-02-11 19:55:54,769 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67-top 2023-02-11 19:55:54,772 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb-top 2023-02-11 19:55:54,775 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568-top 2023-02-11 19:55:54,778 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259-top 2023-02-11 19:55:54,782 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73-top 2023-02-11 19:55:54,785 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6-top 2023-02-11 19:55:54,788 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b-top 2023-02-11 19:55:54,788 INFO [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(310): Store=59d5d182806612f0afda82c67a7a2773/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:54,789 INFO [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:54,789 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2 2023-02-11 19:55:54,789 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2 2023-02-11 19:55:54,790 INFO [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 59d5d182806612f0afda82c67a7a2773 columnFamilyName cf2 2023-02-11 19:55:54,800 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f-top 2023-02-11 19:55:54,803 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878-top 2023-02-11 19:55:54,806 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20-top 2023-02-11 19:55:54,809 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25-top 2023-02-11 19:55:54,812 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8-top 2023-02-11 19:55:54,815 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698-top 2023-02-11 19:55:54,818 DEBUG [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37-top 2023-02-11 19:55:54,818 INFO [StoreOpener-59d5d182806612f0afda82c67a7a2773-1] regionserver.HStore(310): Store=59d5d182806612f0afda82c67a7a2773/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:54,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:54,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:54,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:54,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:54,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 59d5d182806612f0afda82c67a7a2773; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70056423, jitterRate=0.043922051787376404}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:54,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 59d5d182806612f0afda82c67a7a2773: Running coprocessor pre-open hook at 1676145354756Writing region info on filesystem at 1676145354756Initializing all the Stores at 1676145354757 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145354757Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145354757Cleaning up temporary data from old regions at 1676145354820 (+63 ms)Cleaning up detritus from prior splits at 1676145354820Running coprocessor post-open hooks at 1676145354822 (+2 ms)Region opened successfully at 1676145354822 2023-02-11 19:55:54,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773., pid=150, masterSystemTime=1676145354753 2023-02-11 19:55:54,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:54,824 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:54,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:54,824 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:54,824 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:54,824 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): 59d5d182806612f0afda82c67a7a2773/cf1 is initiating minor compaction (all files) 2023-02-11 19:55:54,825 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:54,825 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): 59d5d182806612f0afda82c67a7a2773/cf2 is initiating minor compaction (all files) 2023-02-11 19:55:54,825 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 59d5d182806612f0afda82c67a7a2773/cf1 in testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:54,825 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 59d5d182806612f0afda82c67a7a2773/cf2 in testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:54,825 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878-top] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/.tmp, totalSize=34.9 K 2023-02-11 19:55:54,825 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb-top] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/.tmp, totalSize=34.9 K 2023-02-11 19:55:54,826 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145351073 2023-02-11 19:55:54,826 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676145351073 2023-02-11 19:55:54,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:54,826 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145351635 2023-02-11 19:55:54,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:54,826 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676145351635 2023-02-11 19:55:54,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:54,826 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=59d5d182806612f0afda82c67a7a2773, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:54,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 29bf5a2ffc186ac6925edb613a6518ba, NAME => 'testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 19:55:54,826 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145352215 2023-02-11 19:55:54,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145354826"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145354826"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145354826"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145354826"}]},"ts":"1676145354826"} 2023-02-11 19:55:54,826 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676145352215 2023-02-11 19:55:54,827 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676145352397 2023-02-11 19:55:54,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 19:55:54,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:54,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 19:55:54,827 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676145352397 2023-02-11 19:55:54,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:54,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:54,827 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676145352578 2023-02-11 19:55:54,828 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676145352578 2023-02-11 19:55:54,828 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676145352740 2023-02-11 19:55:54,828 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676145352740 2023-02-11 19:55:54,829 INFO [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:54,829 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145352915 2023-02-11 19:55:54,829 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676145352915 2023-02-11 19:55:54,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-11 19:55:54,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 59d5d182806612f0afda82c67a7a2773, server=jenkins-hbase4.apache.org,42733,1676145241889 in 223 msec 2023-02-11 19:55:54,831 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1 2023-02-11 19:55:54,831 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1 2023-02-11 19:55:54,831 INFO [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29bf5a2ffc186ac6925edb613a6518ba columnFamilyName cf1 2023-02-11 19:55:54,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=59d5d182806612f0afda82c67a7a2773, ASSIGN in 382 msec 2023-02-11 19:55:54,848 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): 59d5d182806612f0afda82c67a7a2773#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:54,848 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): 59d5d182806612f0afda82c67a7a2773#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:54,851 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67-bottom 2023-02-11 19:55:54,858 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb-bottom 2023-02-11 19:55:54,861 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568-bottom 2023-02-11 19:55:54,869 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259-bottom 2023-02-11 19:55:54,871 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/.tmp/cf2/82d1f2818fcd4b67bf1bf3f4d71169e6 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/82d1f2818fcd4b67bf1bf3f4d71169e6 2023-02-11 19:55:54,872 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/.tmp/cf1/48f918e709bc41caa8975a947cac25ac as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/48f918e709bc41caa8975a947cac25ac 2023-02-11 19:55:54,872 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73-bottom 2023-02-11 19:55:54,876 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 59d5d182806612f0afda82c67a7a2773/cf2 of 59d5d182806612f0afda82c67a7a2773 into 82d1f2818fcd4b67bf1bf3f4d71169e6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:54,876 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 59d5d182806612f0afda82c67a7a2773/cf1 of 59d5d182806612f0afda82c67a7a2773 into 48f918e709bc41caa8975a947cac25ac(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:54,876 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 59d5d182806612f0afda82c67a7a2773: 2023-02-11 19:55:54,876 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6-bottom 2023-02-11 19:55:54,876 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 59d5d182806612f0afda82c67a7a2773: 2023-02-11 19:55:54,876 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773., storeName=59d5d182806612f0afda82c67a7a2773/cf1, priority=9, startTime=1676145354823; duration=0sec 2023-02-11 19:55:54,876 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773., storeName=59d5d182806612f0afda82c67a7a2773/cf2, priority=9, startTime=1676145354824; duration=0sec 2023-02-11 19:55:54,876 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:54,876 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:54,879 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b-bottom 2023-02-11 19:55:54,879 INFO [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(310): Store=29bf5a2ffc186ac6925edb613a6518ba/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:54,879 INFO [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:54,879 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2 2023-02-11 19:55:54,880 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2 2023-02-11 19:55:54,880 INFO [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29bf5a2ffc186ac6925edb613a6518ba columnFamilyName cf2 2023-02-11 19:55:54,889 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f-bottom 2023-02-11 19:55:54,891 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878-bottom 2023-02-11 19:55:54,893 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20-bottom 2023-02-11 19:55:54,896 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25-bottom 2023-02-11 19:55:54,899 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8-bottom 2023-02-11 19:55:54,902 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698-bottom 2023-02-11 19:55:54,904 DEBUG [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(539): loaded hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37-bottom 2023-02-11 19:55:54,905 INFO [StoreOpener-29bf5a2ffc186ac6925edb613a6518ba-1] regionserver.HStore(310): Store=29bf5a2ffc186ac6925edb613a6518ba/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 19:55:54,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:54,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:54,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 19:55:54,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:54,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 29bf5a2ffc186ac6925edb613a6518ba; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68041984, jitterRate=0.013904571533203125}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 19:55:54,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 29bf5a2ffc186ac6925edb613a6518ba: Running coprocessor pre-open hook at 1676145354828Writing region info on filesystem at 1676145354828Initializing all the Stores at 1676145354828Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145354828Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676145354828Cleaning up temporary data from old regions at 1676145354906 (+78 ms)Cleaning up detritus from prior splits at 1676145354907 (+1 ms)Running coprocessor post-open hooks at 1676145354908 (+1 ms)Region opened successfully at 1676145354908 2023-02-11 19:55:54,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba., pid=149, masterSystemTime=1676145354753 2023-02-11 19:55:54,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:54,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 19:55:54,910 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:54,910 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 19:55:54,910 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:54,910 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1912): 29bf5a2ffc186ac6925edb613a6518ba/cf2 is initiating minor compaction (all files) 2023-02-11 19:55:54,910 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:54,910 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1912): 29bf5a2ffc186ac6925edb613a6518ba/cf1 is initiating minor compaction (all files) 2023-02-11 19:55:54,910 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 29bf5a2ffc186ac6925edb613a6518ba/cf2 in testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:54,910 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 29bf5a2ffc186ac6925edb613a6518ba/cf1 in testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:54,911 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/.tmp, totalSize=34.9 K 2023-02-11 19:55:54,911 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb-bottom] into tmpdir=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/.tmp, totalSize=34.9 K 2023-02-11 19:55:54,911 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145351073 2023-02-11 19:55:54,911 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676145351073 2023-02-11 19:55:54,911 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145351635 2023-02-11 19:55:54,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:54,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:54,912 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676145351635 2023-02-11 19:55:54,912 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=29bf5a2ffc186ac6925edb613a6518ba, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:54,912 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145352215 2023-02-11 19:55:54,912 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145354912"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676145354912"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676145354912"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676145354912"}]},"ts":"1676145354912"} 2023-02-11 19:55:54,913 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676145352215 2023-02-11 19:55:54,913 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145352397 2023-02-11 19:55:54,913 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676145352397 2023-02-11 19:55:54,913 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145352578 2023-02-11 19:55:54,914 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676145352578 2023-02-11 19:55:54,914 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145352740 2023-02-11 19:55:54,914 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676145352740 2023-02-11 19:55:54,914 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] compactions.Compactor(207): Compacting 22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145352915 2023-02-11 19:55:54,915 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] compactions.Compactor(207): Compacting 837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676145352915 2023-02-11 19:55:54,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-11 19:55:54,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 29bf5a2ffc186ac6925edb613a6518ba, server=jenkins-hbase4.apache.org,42733,1676145241889 in 311 msec 2023-02-11 19:55:54,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-11 19:55:54,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=29bf5a2ffc186ac6925edb613a6518ba, ASSIGN in 468 msec 2023-02-11 19:55:54,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=b9fef38429861932e92e13f6b792d998, daughterA=29bf5a2ffc186ac6925edb613a6518ba, daughterB=59d5d182806612f0afda82c67a7a2773 in 1.9960 sec 2023-02-11 19:55:54,927 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] throttle.PressureAwareThroughputController(145): 29bf5a2ffc186ac6925edb613a6518ba#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:54,927 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] throttle.PressureAwareThroughputController(145): 29bf5a2ffc186ac6925edb613a6518ba#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 19:55:54,952 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/.tmp/cf1/8cd2624a82a744b493ebdb9df64a4896 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/8cd2624a82a744b493ebdb9df64a4896 2023-02-11 19:55:54,954 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/.tmp/cf2/c22061748a144ea987a98621a1f14383 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c22061748a144ea987a98621a1f14383 2023-02-11 19:55:54,956 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 29bf5a2ffc186ac6925edb613a6518ba/cf1 of 29bf5a2ffc186ac6925edb613a6518ba into 8cd2624a82a744b493ebdb9df64a4896(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:54,956 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 29bf5a2ffc186ac6925edb613a6518ba: 2023-02-11 19:55:54,956 INFO [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba., storeName=29bf5a2ffc186ac6925edb613a6518ba/cf1, priority=9, startTime=1676145354909; duration=0sec 2023-02-11 19:55:54,957 DEBUG [RS:2;jenkins-hbase4:42733-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:54,958 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 29bf5a2ffc186ac6925edb613a6518ba/cf2 of 29bf5a2ffc186ac6925edb613a6518ba into c22061748a144ea987a98621a1f14383(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 19:55:54,958 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 29bf5a2ffc186ac6925edb613a6518ba: 2023-02-11 19:55:54,958 INFO [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba., storeName=29bf5a2ffc186ac6925edb613a6518ba/cf2, priority=9, startTime=1676145354910; duration=0sec 2023-02-11 19:55:54,958 DEBUG [RS:2;jenkins-hbase4:42733-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 19:55:55,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42733] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:45194 deadline: 1676145415173, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998. is not online on jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:55,284 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 19:55:55,288 INFO [Listener at localhost/38771] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 19:55:55,290 DEBUG [Listener at localhost/38771] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 19:55:55,297 WARN [Listener at localhost/38771] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 19:55:55,299 INFO [Listener at localhost/38771] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-11 19:55:55,299 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-11 19:55:55,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-11 19:55:55,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 19:55:55,303 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145355303"}]},"ts":"1676145355303"} 2023-02-11 19:55:55,304 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-11 19:55:55,306 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-11 19:55:55,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=29bf5a2ffc186ac6925edb613a6518ba, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=59d5d182806612f0afda82c67a7a2773, UNASSIGN}] 2023-02-11 19:55:55,307 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=59d5d182806612f0afda82c67a7a2773, UNASSIGN 2023-02-11 19:55:55,308 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=59d5d182806612f0afda82c67a7a2773, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:55,308 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145355308"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145355308"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145355308"}]},"ts":"1676145355308"} 2023-02-11 19:55:55,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 59d5d182806612f0afda82c67a7a2773, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:55,310 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=29bf5a2ffc186ac6925edb613a6518ba, UNASSIGN 2023-02-11 19:55:55,311 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=29bf5a2ffc186ac6925edb613a6518ba, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:55,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145355311"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676145355311"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676145355311"}]},"ts":"1676145355311"} 2023-02-11 19:55:55,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 29bf5a2ffc186ac6925edb613a6518ba, server=jenkins-hbase4.apache.org,42733,1676145241889}] 2023-02-11 19:55:55,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 19:55:55,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:55,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 59d5d182806612f0afda82c67a7a2773, disabling compactions & flushes 2023-02-11 19:55:55,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:55,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:55,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. after waiting 0 ms 2023-02-11 19:55:55,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:55,461 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb-top] to archive 2023-02-11 19:55:55,462 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:55,463 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,464 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,465 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,466 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,467 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,468 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,469 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,470 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698-top, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878-top] to archive 2023-02-11 19:55:55,470 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:55,471 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,472 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,473 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,474 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,475 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,476 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,477 DEBUG [StoreCloser-testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:55:55,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:55,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773. 2023-02-11 19:55:55,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 59d5d182806612f0afda82c67a7a2773: Waiting for close lock at 1676145355460Running coprocessor pre-close hooks at 1676145355460Disabling compacts and flushes for region at 1676145355460Disabling writes for close at 1676145355460Writing region close event to WAL at 1676145355478 (+18 ms)Running coprocessor post-close hooks at 1676145355480 (+2 ms)Closed at 1676145355480 2023-02-11 19:55:55,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:55,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:55,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 29bf5a2ffc186ac6925edb613a6518ba, disabling compactions & flushes 2023-02-11 19:55:55,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:55,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:55,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. after waiting 0 ms 2023-02-11 19:55:55,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:55,482 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=59d5d182806612f0afda82c67a7a2773, regionState=CLOSED 2023-02-11 19:55:55,482 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145355482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145355482"}]},"ts":"1676145355482"} 2023-02-11 19:55:55,482 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb-bottom] to archive 2023-02-11 19:55:55,483 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:55,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-11 19:55:55,484 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/eea19316baea406491729f2686e1656b.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 59d5d182806612f0afda82c67a7a2773, server=jenkins-hbase4.apache.org,42733,1676145241889 in 174 msec 2023-02-11 19:55:55,485 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a3738cce04ca43dca5f03e7583d1c259.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=59d5d182806612f0afda82c67a7a2773, UNASSIGN in 177 msec 2023-02-11 19:55:55,486 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a32d7b2977af4a5ea32db70cd3a1c568.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,487 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/ae7da5799ee44558b2f56ded9ca967d6.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,488 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/a781238f991b460494baf8c420770f73.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,489 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/0bb6af9ef2b5464ba2eada69a97e6d67.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,490 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/837659bd108742bb9977478deedabbdb.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,491 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698-bottom, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998->hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878-bottom] to archive 2023-02-11 19:55:55,492 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 19:55:55,493 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/1b9b9e1f27914d93a33823ff7167592f.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,494 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/a5ab8f5bca004d23ab463361642db3d8.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,494 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c2b4237dd67e4095b293a62013077a37.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,495 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/5f7c838085b94e48b651b6bb13b78d25.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,496 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/4c37e969e3d74ee7999315a7d0532a20.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,497 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c11361d8e1034bcabf76a719b5fdc698.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,498 DEBUG [StoreCloser-testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/22f30b5e76984cad933a0ec50fc8d878.b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 19:55:55,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:55,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba. 2023-02-11 19:55:55,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 29bf5a2ffc186ac6925edb613a6518ba: Waiting for close lock at 1676145355481Running coprocessor pre-close hooks at 1676145355481Disabling compacts and flushes for region at 1676145355481Disabling writes for close at 1676145355481Writing region close event to WAL at 1676145355498 (+17 ms)Running coprocessor post-close hooks at 1676145355501 (+3 ms)Closed at 1676145355501 2023-02-11 19:55:55,502 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:55,502 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=29bf5a2ffc186ac6925edb613a6518ba, regionState=CLOSED 2023-02-11 19:55:55,503 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676145355502"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676145355502"}]},"ts":"1676145355502"} 2023-02-11 19:55:55,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-11 19:55:55,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 29bf5a2ffc186ac6925edb613a6518ba, server=jenkins-hbase4.apache.org,42733,1676145241889 in 192 msec 2023-02-11 19:55:55,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-11 19:55:55,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=29bf5a2ffc186ac6925edb613a6518ba, UNASSIGN in 198 msec 2023-02-11 19:55:55,506 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676145355506"}]},"ts":"1676145355506"} 2023-02-11 19:55:55,507 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-11 19:55:55,508 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-11 19:55:55,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 210 msec 2023-02-11 19:55:55,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 19:55:55,604 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-11 19:55:55,605 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-11 19:55:55,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 19:55:55,607 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 19:55:55,607 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 19:55:55,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-11 19:55:55,611 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,611 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:55,611 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:55,613 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/recovered.edits] 2023-02-11 19:55:55,613 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/recovered.edits] 2023-02-11 19:55:55,613 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2, FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/recovered.edits] 2023-02-11 19:55:55,615 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/48f918e709bc41caa8975a947cac25ac to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf1/48f918e709bc41caa8975a947cac25ac 2023-02-11 19:55:55,615 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/8cd2624a82a744b493ebdb9df64a4896 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf1/8cd2624a82a744b493ebdb9df64a4896 2023-02-11 19:55:55,616 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/0bb6af9ef2b5464ba2eada69a97e6d67 2023-02-11 19:55:55,617 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/82d1f2818fcd4b67bf1bf3f4d71169e6 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/cf2/82d1f2818fcd4b67bf1bf3f4d71169e6 2023-02-11 19:55:55,617 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c22061748a144ea987a98621a1f14383 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/cf2/c22061748a144ea987a98621a1f14383 2023-02-11 19:55:55,618 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/837659bd108742bb9977478deedabbdb 2023-02-11 19:55:55,619 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a32d7b2977af4a5ea32db70cd3a1c568 2023-02-11 19:55:55,620 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba/recovered.edits/90.seqid 2023-02-11 19:55:55,620 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/recovered.edits/90.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773/recovered.edits/90.seqid 2023-02-11 19:55:55,620 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a3738cce04ca43dca5f03e7583d1c259 2023-02-11 19:55:55,620 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/29bf5a2ffc186ac6925edb613a6518ba 2023-02-11 19:55:55,621 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/59d5d182806612f0afda82c67a7a2773 2023-02-11 19:55:55,621 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/a781238f991b460494baf8c420770f73 2023-02-11 19:55:55,622 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/ae7da5799ee44558b2f56ded9ca967d6 2023-02-11 19:55:55,623 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf1/eea19316baea406491729f2686e1656b 2023-02-11 19:55:55,625 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/1b9b9e1f27914d93a33823ff7167592f 2023-02-11 19:55:55,626 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/22f30b5e76984cad933a0ec50fc8d878 2023-02-11 19:55:55,627 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/4c37e969e3d74ee7999315a7d0532a20 2023-02-11 19:55:55,628 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/5f7c838085b94e48b651b6bb13b78d25 2023-02-11 19:55:55,629 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/a5ab8f5bca004d23ab463361642db3d8 2023-02-11 19:55:55,630 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c11361d8e1034bcabf76a719b5fdc698 2023-02-11 19:55:55,631 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37 to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/cf2/c2b4237dd67e4095b293a62013077a37 2023-02-11 19:55:55,632 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/recovered.edits/85.seqid to hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/archive/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998/recovered.edits/85.seqid 2023-02-11 19:55:55,633 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/.tmp/data/default/testSplitTableRegion/b9fef38429861932e92e13f6b792d998 2023-02-11 19:55:55,633 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-11 19:55:55,634 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 19:55:55,636 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-11 19:55:55,637 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-11 19:55:55,638 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 19:55:55,638 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-11 19:55:55,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145355638"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:55,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145355638"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:55,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676145355638"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:55,640 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 19:55:55,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b9fef38429861932e92e13f6b792d998, NAME => 'testSplitTableRegion,,1676145350070.b9fef38429861932e92e13f6b792d998.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 29bf5a2ffc186ac6925edb613a6518ba, NAME => 'testSplitTableRegion,,1676145352922.29bf5a2ffc186ac6925edb613a6518ba.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 59d5d182806612f0afda82c67a7a2773, NAME => 'testSplitTableRegion,41,1676145352922.59d5d182806612f0afda82c67a7a2773.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 19:55:55,640 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-11 19:55:55,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676145355640"}]},"ts":"9223372036854775807"} 2023-02-11 19:55:55,641 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-11 19:55:55,643 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 19:55:55,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 38 msec 2023-02-11 19:55:55,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37191] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-11 19:55:55,709 INFO [Listener at localhost/38771] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-11 19:55:55,722 INFO [Listener at localhost/38771] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374 (was 373) - Thread LEAK? -, OpenFileDescriptor=824 (was 841), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=200 (was 149) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3362 (was 3368) 2023-02-11 19:55:55,722 INFO [Listener at localhost/38771] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-11 19:55:55,722 INFO [Listener at localhost/38771] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 19:55:55,724 DEBUG [Listener at localhost/38771] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x65cede91 to 127.0.0.1:50506 2023-02-11 19:55:55,724 DEBUG [Listener at localhost/38771] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:55,889 DEBUG [Listener at localhost/38771] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-11 19:55:55,889 DEBUG [Listener at localhost/38771] util.JVMClusterUtil(257): Found active master hash=1951274840, stopped=false 2023-02-11 19:55:55,889 INFO [Listener at localhost/38771] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:55:55,890 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 19:55:55,890 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 19:55:55,890 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 19:55:55,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 19:55:55,891 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:55:55,891 INFO [Listener at localhost/38771] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 19:55:55,891 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 19:55:55,891 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 19:55:55,891 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 19:55:55,891 DEBUG [Listener at localhost/38771] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29489d83 to 127.0.0.1:50506 2023-02-11 19:55:55,891 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 19:55:55,891 DEBUG [Listener at localhost/38771] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:55,892 INFO [Listener at localhost/38771] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,32837,1676145241806' ***** 2023-02-11 19:55:55,892 INFO [Listener at localhost/38771] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 19:55:55,892 INFO [Listener at localhost/38771] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43747,1676145241852' ***** 2023-02-11 19:55:55,892 INFO [Listener at localhost/38771] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 19:55:55,892 INFO [Listener at localhost/38771] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42733,1676145241889' ***** 2023-02-11 19:55:55,892 INFO [Listener at localhost/38771] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 19:55:55,892 INFO [RS:0;jenkins-hbase4:32837] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 19:55:55,892 INFO [RS:2;jenkins-hbase4:42733] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 19:55:55,892 INFO [RS:0;jenkins-hbase4:32837] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 19:55:55,892 INFO [RS:0;jenkins-hbase4:32837] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 19:55:55,892 INFO [RS:1;jenkins-hbase4:43747] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 19:55:55,893 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:55,893 INFO [RS:1;jenkins-hbase4:43747] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 19:55:55,892 INFO [RS:2;jenkins-hbase4:42733] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 19:55:55,893 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 19:55:55,893 INFO [RS:2;jenkins-hbase4:42733] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 19:55:55,892 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 19:55:55,892 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 19:55:55,893 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(3304): Received CLOSE for 6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:55:55,893 INFO [RS:1;jenkins-hbase4:43747] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 19:55:55,893 DEBUG [RS:0;jenkins-hbase4:32837] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x45cef5bf to 127.0.0.1:50506 2023-02-11 19:55:55,893 INFO [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:55,893 DEBUG [RS:0;jenkins-hbase4:32837] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:55,893 DEBUG [RS:1;jenkins-hbase4:43747] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x471ef5a4 to 127.0.0.1:50506 2023-02-11 19:55:55,893 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:55,893 DEBUG [RS:1;jenkins-hbase4:43747] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:55,893 INFO [RS:0;jenkins-hbase4:32837] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 19:55:55,894 INFO [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43747,1676145241852; all regions closed. 2023-02-11 19:55:55,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6558cedf10894666d9fab2b1f6f4142d, disabling compactions & flushes 2023-02-11 19:55:55,893 DEBUG [RS:2;jenkins-hbase4:42733] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e0340aa to 127.0.0.1:50506 2023-02-11 19:55:55,894 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:55:55,894 DEBUG [RS:2;jenkins-hbase4:42733] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:55,894 INFO [RS:0;jenkins-hbase4:32837] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 19:55:55,894 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-11 19:55:55,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:55:55,894 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1479): Online Regions={6558cedf10894666d9fab2b1f6f4142d=hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d.} 2023-02-11 19:55:55,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. after waiting 0 ms 2023-02-11 19:55:55,894 INFO [RS:0;jenkins-hbase4:32837] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 19:55:55,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:55:55,894 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-11 19:55:55,894 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 6558cedf10894666d9fab2b1f6f4142d 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-11 19:55:55,895 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-11 19:55:55,895 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-11 19:55:55,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 19:55:55,895 DEBUG [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1505): Waiting on 6558cedf10894666d9fab2b1f6f4142d 2023-02-11 19:55:55,895 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 19:55:55,895 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 19:55:55,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 19:55:55,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 19:55:55,895 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 19:55:55,895 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-11 19:55:55,935 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d/.tmp/info/3361f4e4add444a99c2d8b8afe906409 2023-02-11 19:55:55,936 DEBUG [RS:1;jenkins-hbase4:43747] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/oldWALs 2023-02-11 19:55:55,936 INFO [RS:1;jenkins-hbase4:43747] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43747%2C1676145241852:(num 1676145243312) 2023-02-11 19:55:55,936 DEBUG [RS:1;jenkins-hbase4:43747] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:55,936 INFO [RS:1;jenkins-hbase4:43747] regionserver.LeaseManager(133): Closed leases 2023-02-11 19:55:55,937 INFO [RS:1;jenkins-hbase4:43747] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-11 19:55:55,937 INFO [RS:1;jenkins-hbase4:43747] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 19:55:55,937 INFO [RS:1;jenkins-hbase4:43747] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 19:55:55,937 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 19:55:55,937 INFO [RS:1;jenkins-hbase4:43747] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 19:55:55,938 INFO [RS:1;jenkins-hbase4:43747] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43747 2023-02-11 19:55:55,944 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:55,944 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:55,944 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 19:55:55,944 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43747,1676145241852 2023-02-11 19:55:55,944 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 19:55:55,944 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 19:55:55,945 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 19:55:55,945 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@696f3c22 rejected from java.util.concurrent.ThreadPoolExecutor@6d50b5d7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:55,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d/.tmp/info/3361f4e4add444a99c2d8b8afe906409 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d/info/3361f4e4add444a99c2d8b8afe906409 2023-02-11 19:55:55,947 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43747,1676145241852] 2023-02-11 19:55:55,947 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43747,1676145241852; numProcessing=1 2023-02-11 19:55:55,948 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43747,1676145241852 already deleted, retry=false 2023-02-11 19:55:55,948 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43747,1676145241852 expired; onlineServers=2 2023-02-11 19:55:55,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d/info/3361f4e4add444a99c2d8b8afe906409, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 19:55:55,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6558cedf10894666d9fab2b1f6f4142d in 57ms, sequenceid=6, compaction requested=false 2023-02-11 19:55:55,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/namespace/6558cedf10894666d9fab2b1f6f4142d/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-11 19:55:55,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:55,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:55:55,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6558cedf10894666d9fab2b1f6f4142d: Waiting for close lock at 1676145355894Running coprocessor pre-close hooks at 1676145355894Disabling compacts and flushes for region at 1676145355894Disabling writes for close at 1676145355894Obtaining lock to block concurrent updates at 1676145355894Preparing flush snapshotting stores in 6558cedf10894666d9fab2b1f6f4142d at 1676145355894Finished memstore snapshotting hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676145355895 (+1 ms)Flushing stores of hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. at 1676145355896 (+1 ms)Flushing 6558cedf10894666d9fab2b1f6f4142d/info: creating writer at 1676145355896Flushing 6558cedf10894666d9fab2b1f6f4142d/info: appending metadata at 1676145355907 (+11 ms)Flushing 6558cedf10894666d9fab2b1f6f4142d/info: closing flushed file at 1676145355907Flushing 6558cedf10894666d9fab2b1f6f4142d/info: reopening flushed file at 1676145355947 (+40 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6558cedf10894666d9fab2b1f6f4142d in 57ms, sequenceid=6, compaction requested=false at 1676145355951 (+4 ms)Writing region close event to WAL at 1676145355952 (+1 ms)Running coprocessor post-close hooks at 1676145355954 (+2 ms)Closed at 1676145355955 (+1 ms) 2023-02-11 19:55:55,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676145243808.6558cedf10894666d9fab2b1f6f4142d. 2023-02-11 19:55:55,987 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 19:55:55,987 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 19:55:55,987 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 19:55:56,047 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 19:55:56,047 INFO [RS:1;jenkins-hbase4:43747] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43747,1676145241852; zookeeper connection closed. 2023-02-11 19:55:56,047 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@9ae8852 rejected from java.util.concurrent.ThreadPoolExecutor@6d50b5d7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,047 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:43747-0x10044f292690002, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 19:55:56,047 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7126db95 rejected from java.util.concurrent.ThreadPoolExecutor@6d50b5d7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,047 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7bf8f723] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7bf8f723 2023-02-11 19:55:56,095 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42733,1676145241889; all regions closed. 2023-02-11 19:55:56,095 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 19:55:56,100 DEBUG [RS:2;jenkins-hbase4:42733] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/oldWALs 2023-02-11 19:55:56,100 INFO [RS:2;jenkins-hbase4:42733] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42733%2C1676145241889:(num 1676145243312) 2023-02-11 19:55:56,100 DEBUG [RS:2;jenkins-hbase4:42733] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:56,100 INFO [RS:2;jenkins-hbase4:42733] regionserver.LeaseManager(133): Closed leases 2023-02-11 19:55:56,100 INFO [RS:2;jenkins-hbase4:42733] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-11 19:55:56,101 INFO [RS:2;jenkins-hbase4:42733] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 19:55:56,101 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 19:55:56,101 INFO [RS:2;jenkins-hbase4:42733] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 19:55:56,101 INFO [RS:2;jenkins-hbase4:42733] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 19:55:56,102 INFO [RS:2;jenkins-hbase4:42733] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42733 2023-02-11 19:55:56,103 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:56,103 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 19:55:56,103 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@b2765b3 rejected from java.util.concurrent.ThreadPoolExecutor@f18f00f[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,103 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42733,1676145241889 2023-02-11 19:55:56,104 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42733,1676145241889] 2023-02-11 19:55:56,104 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42733,1676145241889; numProcessing=2 2023-02-11 19:55:56,106 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42733,1676145241889 already deleted, retry=false 2023-02-11 19:55:56,106 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42733,1676145241889 expired; onlineServers=1 2023-02-11 19:55:56,200 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-11 19:55:56,200 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-11 19:55:56,204 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 19:55:56,204 INFO [RS:2;jenkins-hbase4:42733] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42733,1676145241889; zookeeper connection closed. 2023-02-11 19:55:56,204 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4b202ed7 rejected from java.util.concurrent.ThreadPoolExecutor@f18f00f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,204 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:42733-0x10044f292690003, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 19:55:56,204 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@375766dc] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@375766dc 2023-02-11 19:55:56,204 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@334adf09 rejected from java.util.concurrent.ThreadPoolExecutor@f18f00f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,295 DEBUG [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 19:55:56,343 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/.tmp/info/a771e0ddfcd24a50adcfa29cee343aed 2023-02-11 19:55:56,347 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a771e0ddfcd24a50adcfa29cee343aed 2023-02-11 19:55:56,354 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/.tmp/rep_barrier/26b3c9b3c6034b48a9c8a852e671523b 2023-02-11 19:55:56,357 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 26b3c9b3c6034b48a9c8a852e671523b 2023-02-11 19:55:56,368 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/.tmp/table/f2ad4cfe7f584ca494475c05ba3c4e07 2023-02-11 19:55:56,371 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f2ad4cfe7f584ca494475c05ba3c4e07 2023-02-11 19:55:56,372 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/.tmp/info/a771e0ddfcd24a50adcfa29cee343aed as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/info/a771e0ddfcd24a50adcfa29cee343aed 2023-02-11 19:55:56,375 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a771e0ddfcd24a50adcfa29cee343aed 2023-02-11 19:55:56,375 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/info/a771e0ddfcd24a50adcfa29cee343aed, entries=59, sequenceid=210, filesize=11.1 K 2023-02-11 19:55:56,375 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/.tmp/rep_barrier/26b3c9b3c6034b48a9c8a852e671523b as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/rep_barrier/26b3c9b3c6034b48a9c8a852e671523b 2023-02-11 19:55:56,378 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 26b3c9b3c6034b48a9c8a852e671523b 2023-02-11 19:55:56,378 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/rep_barrier/26b3c9b3c6034b48a9c8a852e671523b, entries=24, sequenceid=210, filesize=7.5 K 2023-02-11 19:55:56,379 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/.tmp/table/f2ad4cfe7f584ca494475c05ba3c4e07 as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/table/f2ad4cfe7f584ca494475c05ba3c4e07 2023-02-11 19:55:56,382 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f2ad4cfe7f584ca494475c05ba3c4e07 2023-02-11 19:55:56,382 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/table/f2ad4cfe7f584ca494475c05ba3c4e07, entries=36, sequenceid=210, filesize=8.0 K 2023-02-11 19:55:56,382 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 487ms, sequenceid=210, compaction requested=false 2023-02-11 19:55:56,386 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-11 19:55:56,386 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 19:55:56,386 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 19:55:56,387 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 19:55:56,387 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676145355895Running coprocessor pre-close hooks at 1676145355895Disabling compacts and flushes for region at 1676145355895Disabling writes for close at 1676145355895Obtaining lock to block concurrent updates at 1676145355895Preparing flush snapshotting stores in 1588230740 at 1676145355895Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676145355895Flushing stores of hbase:meta,,1.1588230740 at 1676145355896 (+1 ms)Flushing 1588230740/info: creating writer at 1676145355896Flushing 1588230740/info: appending metadata at 1676145355914 (+18 ms)Flushing 1588230740/info: closing flushed file at 1676145355914Flushing 1588230740/rep_barrier: creating writer at 1676145356347 (+433 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676145356349 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676145356349Flushing 1588230740/table: creating writer at 1676145356357 (+8 ms)Flushing 1588230740/table: appending metadata at 1676145356359 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676145356360 (+1 ms)Flushing 1588230740/info: reopening flushed file at 1676145356372 (+12 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676145356376 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1676145356379 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 487ms, sequenceid=210, compaction requested=false at 1676145356382 (+3 ms)Writing region close event to WAL at 1676145356384 (+2 ms)Running coprocessor post-close hooks at 1676145356386 (+2 ms)Closed at 1676145356387 (+1 ms) 2023-02-11 19:55:56,387 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-11 19:55:56,495 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,32837,1676145241806; all regions closed. 2023-02-11 19:55:56,500 DEBUG [RS:0;jenkins-hbase4:32837] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/oldWALs 2023-02-11 19:55:56,500 INFO [RS:0;jenkins-hbase4:32837] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C32837%2C1676145241806.meta:.meta(num 1676145243508) 2023-02-11 19:55:56,504 DEBUG [RS:0;jenkins-hbase4:32837] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/oldWALs 2023-02-11 19:55:56,504 INFO [RS:0;jenkins-hbase4:32837] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C32837%2C1676145241806:(num 1676145243312) 2023-02-11 19:55:56,504 DEBUG [RS:0;jenkins-hbase4:32837] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:56,504 INFO [RS:0;jenkins-hbase4:32837] regionserver.LeaseManager(133): Closed leases 2023-02-11 19:55:56,504 INFO [RS:0;jenkins-hbase4:32837] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 19:55:56,504 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 19:55:56,505 INFO [RS:0;jenkins-hbase4:32837] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:32837 2023-02-11 19:55:56,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,32837,1676145241806 2023-02-11 19:55:56,507 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 19:55:56,508 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,32837,1676145241806] 2023-02-11 19:55:56,508 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,32837,1676145241806; numProcessing=3 2023-02-11 19:55:56,509 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,32837,1676145241806 already deleted, retry=false 2023-02-11 19:55:56,509 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,32837,1676145241806 expired; onlineServers=0 2023-02-11 19:55:56,509 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37191,1676145240670' ***** 2023-02-11 19:55:56,509 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-11 19:55:56,510 DEBUG [M:0;jenkins-hbase4:37191] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35731d63, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 19:55:56,510 INFO [M:0;jenkins-hbase4:37191] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37191,1676145240670 2023-02-11 19:55:56,510 INFO [M:0;jenkins-hbase4:37191] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37191,1676145240670; all regions closed. 2023-02-11 19:55:56,510 DEBUG [M:0;jenkins-hbase4:37191] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 19:55:56,510 DEBUG [M:0;jenkins-hbase4:37191] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-11 19:55:56,510 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-11 19:55:56,510 DEBUG [M:0;jenkins-hbase4:37191] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-11 19:55:56,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676145242963] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676145242963,5,FailOnTimeoutGroup] 2023-02-11 19:55:56,510 INFO [M:0;jenkins-hbase4:37191] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-11 19:55:56,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676145242963] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676145242963,5,FailOnTimeoutGroup] 2023-02-11 19:55:56,510 INFO [M:0;jenkins-hbase4:37191] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-11 19:55:56,511 INFO [M:0;jenkins-hbase4:37191] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-11 19:55:56,511 DEBUG [M:0;jenkins-hbase4:37191] master.HMaster(1502): Stopping service threads 2023-02-11 19:55:56,511 INFO [M:0;jenkins-hbase4:37191] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-11 19:55:56,511 ERROR [M:0;jenkins-hbase4:37191] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-11 19:55:56,511 INFO [M:0;jenkins-hbase4:37191] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 19:55:56,511 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-11 19:55:56,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 19:55:56,512 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 19:55:56,512 DEBUG [M:0;jenkins-hbase4:37191] zookeeper.ZKUtil(398): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-11 19:55:56,512 WARN [M:0;jenkins-hbase4:37191] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-11 19:55:56,512 INFO [M:0;jenkins-hbase4:37191] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 19:55:56,512 INFO [M:0;jenkins-hbase4:37191] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-11 19:55:56,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 19:55:56,512 DEBUG [M:0;jenkins-hbase4:37191] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 19:55:56,512 INFO [M:0;jenkins-hbase4:37191] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 19:55:56,512 DEBUG [M:0;jenkins-hbase4:37191] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 19:55:56,512 DEBUG [M:0;jenkins-hbase4:37191] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 19:55:56,513 DEBUG [M:0;jenkins-hbase4:37191] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 19:55:56,513 INFO [M:0;jenkins-hbase4:37191] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.94 KB heapSize=718.96 KB 2023-02-11 19:55:56,522 INFO [M:0;jenkins-hbase4:37191] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.94 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/70134ba27bbe4e739b4ba2775debab7a 2023-02-11 19:55:56,525 INFO [M:0;jenkins-hbase4:37191] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70134ba27bbe4e739b4ba2775debab7a 2023-02-11 19:55:56,526 DEBUG [M:0;jenkins-hbase4:37191] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/70134ba27bbe4e739b4ba2775debab7a as hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/70134ba27bbe4e739b4ba2775debab7a 2023-02-11 19:55:56,528 INFO [M:0;jenkins-hbase4:37191] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70134ba27bbe4e739b4ba2775debab7a 2023-02-11 19:55:56,529 INFO [M:0;jenkins-hbase4:37191] regionserver.HStore(1080): Added hdfs://localhost:42039/user/jenkins/test-data/909b87ba-8d86-61c6-e6df-287f6cfc032f/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/70134ba27bbe4e739b4ba2775debab7a, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-11 19:55:56,529 INFO [M:0;jenkins-hbase4:37191] regionserver.HRegion(2947): Finished flush of dataSize ~607.94 KB/622535, heapSize ~718.95 KB/736200, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 16ms, sequenceid=1282, compaction requested=false 2023-02-11 19:55:56,530 INFO [M:0;jenkins-hbase4:37191] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 19:55:56,530 DEBUG [M:0;jenkins-hbase4:37191] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676145356512Disabling compacts and flushes for region at 1676145356512Disabling writes for close at 1676145356512Obtaining lock to block concurrent updates at 1676145356513 (+1 ms)Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676145356513Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622535, getHeapSize=736200, getOffHeapSize=0, getCellsCount=1499 at 1676145356513Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676145356513Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676145356513Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676145356517 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676145356517Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676145356526 (+9 ms)Finished flush of dataSize ~607.94 KB/622535, heapSize ~718.95 KB/736200, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 16ms, sequenceid=1282, compaction requested=false at 1676145356529 (+3 ms)Writing region close event to WAL at 1676145356530 (+1 ms)Closed at 1676145356530 2023-02-11 19:55:56,533 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 19:55:56,533 INFO [M:0;jenkins-hbase4:37191] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-11 19:55:56,534 INFO [M:0;jenkins-hbase4:37191] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37191 2023-02-11 19:55:56,535 DEBUG [M:0;jenkins-hbase4:37191] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37191,1676145240670 already deleted, retry=false 2023-02-11 19:55:56,608 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 19:55:56,608 INFO [RS:0;jenkins-hbase4:32837] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,32837,1676145241806; zookeeper connection closed. 2023-02-11 19:55:56,608 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@25a1b7e0 rejected from java.util.concurrent.ThreadPoolExecutor@219e378c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 314] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,608 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): regionserver:32837-0x10044f292690001, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 19:55:56,608 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@56ad0235] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@56ad0235 2023-02-11 19:55:56,608 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7d848c08 rejected from java.util.concurrent.ThreadPoolExecutor@219e378c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 314] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,609 INFO [Listener at localhost/38771] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-11 19:55:56,708 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 19:55:56,708 INFO [M:0;jenkins-hbase4:37191] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37191,1676145240670; zookeeper connection closed. 2023-02-11 19:55:56,708 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@40cbc25 rejected from java.util.concurrent.ThreadPoolExecutor@7c6c89af[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,709 DEBUG [Listener at localhost/38771-EventThread] zookeeper.ZKWatcher(600): master:37191-0x10044f292690000, quorum=127.0.0.1:50506, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 19:55:56,709 ERROR [Listener at localhost/38771-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1edd44ac rejected from java.util.concurrent.ThreadPoolExecutor@7c6c89af[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 19:55:56,710 WARN [Listener at localhost/38771] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 19:55:56,713 INFO [Listener at localhost/38771] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 19:55:56,818 WARN [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 2f25de57-daa1-42aa-aa7e-60a3bc1a94d0) service to localhost/127.0.0.1:42039 2023-02-11 19:55:56,820 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data5/current/BP-2104787810-172.31.14.131-1676145237849] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 19:55:56,821 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data6/current/BP-2104787810-172.31.14.131-1676145237849] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 19:55:56,822 WARN [Listener at localhost/38771] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 19:55:56,823 INFO [Listener at localhost/38771] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 19:55:56,930 WARN [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 19:55:56,930 WARN [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 1c013e61-8b4b-4f0a-8efd-0dfb5b64fb4d) service to localhost/127.0.0.1:42039 2023-02-11 19:55:56,931 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data3/current/BP-2104787810-172.31.14.131-1676145237849] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 19:55:56,932 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data4/current/BP-2104787810-172.31.14.131-1676145237849] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 19:55:56,932 WARN [Listener at localhost/38771] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 19:55:56,934 INFO [Listener at localhost/38771] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 19:55:57,036 WARN [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 19:55:57,036 WARN [BP-2104787810-172.31.14.131-1676145237849 heartbeating to localhost/127.0.0.1:42039] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2104787810-172.31.14.131-1676145237849 (Datanode Uuid 1d17abb3-77b3-4fd1-a223-1cc76825d5c0) service to localhost/127.0.0.1:42039 2023-02-11 19:55:57,037 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data1/current/BP-2104787810-172.31.14.131-1676145237849] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 19:55:57,037 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/50e7a776-64cf-c652-9272-e455d3f5c9e4/cluster_67fc1b77-946f-b036-189c-3c8983617de3/dfs/data/data2/current/BP-2104787810-172.31.14.131-1676145237849] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 19:55:57,058 INFO [Listener at localhost/38771] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 19:55:57,246 INFO [Listener at localhost/38771] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-11 19:55:57,484 INFO [Listener at localhost/38771] hbase.HBaseTestingUtility(1293): Minicluster is down